The Recorder app on Pixel sees a 24% increase in engagement with Gemini Nano-powered function

Posted by Terence Zhang – Developer Relations Engineer and Kristi Bradford – Product Supervisor

Google Pixel’s Recorder app permits folks to file, transcribe, save, and share audio. To make it simpler for customers to handle and revisit their recordings, Recorder’s builders turned to Gemini Nano, a robust on-device massive language mannequin (LLM). This integration introduces an AI-powered audio summarization function to assist customers extra simply discover the suitable recordings and shortly grasp key factors.

Earlier this month, Gemini Nano bought an influence increase with the introduction of the brand new Gemini Nano with Multimodality mannequin. The Recorder app is already leveraging this improve to summarize longer voice recordings, with improved processing for grammar and nuance.

Assembly person wants with on-device AI

Recorder builders initially experimented with a cloud-based answer, attaining spectacular ranges of efficiency and high quality. Nonetheless, to prioritize accessibility and privateness for his or her customers, they sought an on-device answer. The event of Gemini Nano introduced an ideal alternative to construct the concise audio summaries customers have been searching for, all whereas retaining knowledge processing on the gadget.

Gemini Nano is Google’s best mannequin for on-device duties. “Having the LLM on-device is helpful to customers as a result of it offers them with extra privateness, much less latency, and it really works wherever they want since there’s no web required,” mentioned Kristi Bradford, the product supervisor for Pixel’s important apps.

To attain higher outcomes, Recorder additionally fine-tuned the mannequin utilizing knowledge that matches its use case. That is executed utilizing low order rank adaptation (LoRA), which allows Gemini Nano to constantly output three-bullet level descriptions of the transcript that embrace any speaker names, key takeaways, and themes.

AICore, an Android system service that centralizes runtime, supply, and demanding security parts for LLMs, considerably streamlined Recorder’s adoption of Gemini Nano. The provision of a developer SDK for operating GenAI workloads allowed the workforce to construct the transcription abstract function in simply 4 months, with solely 4 builders. This effectivity was achieved by eliminating the necessity for sustaining in-house fashions.

Since its launch, Recorder customers have been utilizing the brand new AI-powered summarization function averaging 2 to five instances every day, and the variety of general saved recordings elevated by 24%. This function has contributed to a big enhance in app engagement and person retention general. The Recorder workforce additionally famous that suggestions concerning the new function has been optimistic, with many customers citing the time the brand new AI-powered summarization function saves them.

The following massive evolution: Gemini Nano with multimodality

Recorder builders additionally carried out the newest Gemini Nano mannequin, often known as Gemini Nano with multimodality, to additional enhance its summarization function on Pixel 9 gadgets. The brand new mannequin is considerably bigger than the earlier one on Pixel 8 gadgets, and it’s extra succesful, correct, and scalable. The brand new mannequin additionally has expanded token help that lets Recorder summarize for much longer transcripts than earlier than. Gemini Nano with multimodality is at present solely out there on Pixel 9 gadgets.

Integrating Gemini Nano with multimodality required one other spherical of fine-tuning. Nonetheless, Recorder builders have been in a position to make use of the unique Gemini Nano mannequin’s fine-tuning dataset as a basis, streamlining the event course of.

To completely leverage the brand new mannequin’s capabilities, Recorder builders expanded their dataset with help for longer voice recordings, carried out refined analysis strategies, and established launch standards metrics targeted on grammar and nuance. The inclusion of grammar as a brand new metric for assessing inference high quality was made doable solely by the improved capabilities of Gemini Nano with Multimodality.

Doing extra with on-device AI

“Given the novelty of GenAI, the entire workforce had enjoyable studying tips on how to use it,” mentioned Kristi. “Now, we’re empowered to push the boundaries of what we are able to accomplish whereas assembly rising person wants and alternatives. It’s actually introduced a brand new stage of creativity to problem-solving and experimentation. We’ve already demoed at the very least two extra GenAI options that assist folks get time again internally for early suggestions, and we’re excited concerning the prospects forward.”

Get began

Study extra about tips on how to deliver the advantages of on-device AI with Gemini Nano to your apps.