Android phones with Gemini Nano could gain another handy on-device feature

Big tech companies are racing to create the best generative AI tools for users, developers, and other businesses. Google, for example, offers Gemini, which is both the branding for their AI chatbot as well as the underlying large language model (LLM) that powers it. The Gemini LLM comes in three model sizes: Nano, Pro, and Ultra. Only the Nano model is small enough to run locally on high-end Android devices like the Pixel 8 Pro and the Galaxy S24 series, whereas the other two models run on Google’s cloud servers. Nano’s small size compared to Pro and Ultra means it’s limited in its capabilities, but new evidence suggests this model could gain another interesting feature.

Gemini Nano is only really useful for analyzing or creating small blocks of text. For example, the Nano model currently only powers three AI features on the Pixel 8 Pro: AI summaries of short recordings in the Pixel Recorder app, AI smart replies from Gboard when chatting in WhatsApp, and AI message rewriting suggestions in the Google Messages app. Google’s Gemini Nano model also powers several Galaxy AI features that are available on the Galaxy S24 series, such as Magic Compose.

Because apps can leverage Gemini Nano through an API, it’s easy to add new AI features that rely on it. In fact, evidence seen by Android Authority suggests that Gemini Nano may soon enable AI-powered article summaries. Back in August, Google added a new feature to its experimental Search Generative Experience (SGE) suite that can generate key points for any web page that you’ve opened in the Google app. This feature is available on any Android device provided the user toggles “SGE while browsing” in the Search Labs menu of the Google app.

AI article summaries in the Google app. Credits: Mishaal Rahman

Currently, this AI article summary feature runs on the cloud, which is why it’s available on all devices. Phones with Gemini Nano support like the Pixel 8 Pro and the Galaxy S24 series may soon be able to run this AI article summary feature on-device, if we’re understanding the evidence correctly. To understand the evidence, we first need to briefly explain how Gemini Nano works on Android.

Instead of having apps bundle Gemini Nano on their own, Android’s new AICore service handles the downloading of the model. This cuts down on storage requirements and also simplifies model distribution and updating. Apps can leverage Gemini Nano for on-device inferencing by using a series of APIs provided by Google’s AI Edge SDK. One of these APIs lets apps provide a LoRA (low-rank adaptation) block to fine-tune the Gemini Nano model for a particular task.

AICore’s architecture. Source: Google.

Because machine learning IP and AI safety are so important, Google uses secure downloading APIs to push its Gemini Nano model and LoRA fine-tuning blocks onto devices. These APIs are provided by Android’s Private Compute Services. Private Compute Services is an open-source app that provides APIs for downloading machine learning models from the cloud. It’s part of Android’s Private Compute Core and was created to silo the Android System Intelligence app — which is responsible for many AI-powered features — from the internet.

The architecture of Android’s Private Compute Core. Source: Google.

The API that AICore uses is called Protected Download. Protected Download is an API that “enables downloading of resources to the device with support for a binary transparency log based verification, ensuring these are the official resources provided by Google.” AICore seems to use the Protected Download API to download the Gemini Nano model as well as some LoRA fine-tuning blocks. The AICore app comprises several “clients” of the Protected Download API, and recently, a new “AICore client” called “AI_CORE_CHROME_SUMMARIZATION_OUTPUT” was added.

While the patch that added this “AI_CORE_CHROME_SUMMARIZATION_OUTPUT” client doesn’t have a description that explains its purpose, we’re guessing based on the name and the purpose of the API that the AICore app will soon download a LoRA fine-tuning block that optimizes Gemini Nano for AI article summaries. We could be wrong, though it would make a lot of sense to have Gemini Nano handle AI article summaries on-device. After all, most articles on the web should be short enough for the Gemini Nano model to process. For reference, Gemini Nano is capable of summarizing Pixel Recorder transcripts up to 15 minutes in length.

If we’re right, then we hope that Google announces this feature soon, as the list of on-device AI features that Gemini Nano handles is quite short right now. Since this AI article summary feature is part of the Google app, then we also hope Google enables this on the Galaxy S24 series and not just the Pixel 8 Pro.