# AI on your phone in 2026: what runs offline, and what it takes to get real work done

What offline AI on a phone can and cannot do in 2026, what small local models are good at, and why acting on your accounts is the much harder half.

Canonical: https://openally.ai/blog/ai-on-your-phone-offline
Markdown: https://openally.ai/blog/ai-on-your-phone-offline.md
Published: 2026-08-27
Reviewed: 2026-08-27
Author: Matterward Labs (https://matterwardlabs.com)
Category: Guide
Keywords: run AI on your phone; offline AI app; local AI on phone; offline AI iPhone; local LLM Android; AI that works without internet; on-device AI models
Audience: People who want AI that works without internet; Android and iPhone users comparing offline AI apps; Small business owners deciding what to run on a phone
Disclosure: Matterward Labs makes OpenAlly. Third-party capabilities in this article come from each project's own public documentation and repository, checked on 27 August 2026. Industry figures come from the primary sources linked throughout.

A phone in 2026 can run a real language model with the aeroplane mode switch on. That part is settled. What happens next, when you want the thing to do something other than talk back, is where it gets harder.

## Short answer

A modern phone runs a small language model offline, and it is genuinely useful for private text work: drafting, summarising, sorting, transcribing. Making that same model **act on your accounts**, so it can answer a customer, raise an invoice or move a booking, is a separate and much harder layer. The binding limit is the space the model has left to think in once you have described everything it is allowed to do.

> [!IMPORTANT] Where these claims come from
> Third-party capabilities are taken from each project's own documentation and repository, read on 27 August 2026. OpenAlly device figures come from our own dated field notes on [running Gemma 4 on Android](/blog/gemma-4-android-litert-lm). Product capabilities come from the [OpenAlly feature catalogue](/features).

> Interface view: **Two useful lanes, one deliberate boundary.** Private text work can remain on the phone. Reading or changing a live account is a separate route that needs a connection and access you granted.

## What a phone can actually run today

Offline AI on a phone means the model file sits in the phone's storage and the arithmetic happens on the phone's own chip. No request leaves the handset, so it works on a plane, in a basement, and on a prepaid data plan that ran out on the 20th.

The size band runs from a few hundred megabytes to a few gigabytes. At the small end, Google's AI Edge team documents [Gemma 3 1B at 529MB, reading up to 2,585 tokens per second on a mobile GPU](https://developers.googleblog.com/google-ai-edge-small-language-models-multimodality-rag-function-calling/), which it describes as processing about a page of content in under a second. That figure is the speed of reading a prompt rather than the speed of writing an answer, and the two are far apart on a phone. At the larger end, the two on-device packages in OpenAlly's Android catalogue weigh 2.59GB and 3.66GB, behind device-memory floors of 6GiB and 8GiB (about 6.4GB and 8.6GB), as recorded in our [Gemma 4 field notes](/blog/gemma-4-android-litert-lm).

Storage is the easy half. Installed memory is not available memory: in one load attempt in the same field notes, a 12GB handset reported roughly 5.2GB actually free, and the load failed under memory pressure. That is one observation on one device, not a minimum-RAM rule, and it is why the floors sit so far above the file sizes.

Running a model and letting it call something are also two different pieces of software, shipped on different timetables. Google's on-device function calling library, the piece that lets a local model decide to invoke something, ships as its own component with its [own documentation](https://developers.googleblog.com/google-ai-edge-small-language-models-multimodality-rag-function-calling/).

Hardware matters too. MLC LLM, the engine behind one of the better-known local chat apps, states in its [Android deployment documentation](https://llm.mlc.ai/docs/deploy/android.html) that it needs an actual mobile GPU, and names the Samsung S23 with Snapdragon 8 Gen 2 as the handset its demo build targets. That is a note about a reference build and not a published minimum specification, but it is a fair signal of what these engines are tuned against. Assume a recent phone, and treat an older or cheaper one as something to test.

> Interface view: **A model file can fit before the runtime does.** Published download sizes sit beside one OpenAlly field-note observation: installed memory, actually free memory and supported mobile hardware are different constraints.

## What a local model is genuinely good at

[PocketPal AI](https://github.com/a-ghorbani/pocketpal-ai) runs on Android, iOS and iPadOS, downloads models from Hugging Face, benchmarks them, and works entirely offline once a model is on the device. MLC Chat does the same job on a different engine, compiling models ahead of time for the phone's GPU.

For one class of work they are the right tool:

- **Drafting.** A reply, a description, a first pass at a message you will edit anyway.
- **Summarising.** A long note, a transcript, a document you do not want to paste into a website.
- **Sorting and labelling.** Deciding which of five buckets a piece of text belongs in.
- **Transcription.** Turning a recording into text without the audio leaving the handset. OpenAlly ships this as the [Whisper tool](/features/tools/whisper) on Android, and the [Local LLM tool](/features/tools/local-llm) covers the text side on Android and iPhone.

Every one of those jobs has the same shape. Text goes in, text comes out, and nothing outside the phone needs to change as a result. That shape is what small models are good at, and privacy here is structural, because the request genuinely has nowhere else to go.

> Interface view: **The local sweet spot is a closed text loop.** A note or voice memo becomes a draft, summary, labels or transcript without an account changing elsewhere; OpenAlly Whisper transcription is Android-specific.

## Where local tool use stops

Several local models can use tools already. PocketPal's repository documents a tool-use loop where the model calls built-in engines mid-conversation: `calculate` for arithmetic, `datetime` for the current date and time, and `render_html` for displaying generated markup.

Each of those runs inside the conversation. One evaluates an expression, one reads the clock, one draws markup the model has just written. None of them opens a door to something that belongs to you.

The jobs that give a person hours back are the other kind. Look at the customer message that came in overnight. Check whether that item is in stock. Send the payment link. Move the two o'clock appointment and tell the customer it moved. Each of those has to reach a real account, read the current state of something, and change it. That is a different category of work, and self-contained tools do not reach it.

If what you want is a private assistant that drafts and summarises with no account and no connection, a good local chat app is the correct answer, and you can stop here with the right tool already chosen.

> Interface view: **A self-contained tool stops at the account boundary.** PocketPal's documented calculate, datetime and render_html tools stay inside the conversation. Messages, stock, payments and bookings need another layer.

## Why doing is harder than chatting

Before a model can choose an action, it has to be told what actions exist. Each one arrives as a schema: a name, a description, the arguments it takes, the allowed values. The model reads all of that before it decides anything. On a phone, those descriptions compete for the same small context window as your actual message.

In our own testing, described in the [Gemma 4 field notes](/blog/gemma-4-android-litert-lm), the schemas for fifteen tools alone were estimated at **2,825 tokens**. Against that build's 4,096-token window, the menu was most of the room before the conversation started.

> Interface view: **A 4,096-token window before the conversation starts.** The measured Gemma 4 field-note budget: 2,825 estimated tokens for fifteen tool schemas, leaving the conversation, instructions, history, and answer to share what remains.

The squeeze is not ours alone, and it does not ease at scale. Four groups have published measurements on large tool catalogues, from [Microsoft Research](https://www.microsoft.com/en-us/research/blog/tool-space-interference-in-the-mcp-era-designing-for-agent-compatibility-at-scale/), [GitHub](https://github.blog/ai-and-ml/github-copilot/how-were-making-github-copilot-smarter-with-fewer-tools/), [Anthropic](https://www.anthropic.com/engineering/advanced-tool-use) and a [chance-corrected study from Meta](https://arxiv.org/abs/2605.24660):

| Finding | Reported as | Source |
| --- | --- | --- |
| Large tool spaces can lower model performance for some models | Up to 85% lower | Microsoft Research, 11 September 2025 |
| Public servers already publish catalogues far past what a small window holds | 256 tools in the largest one catalogued | Microsoft Research, 11 September 2025 |
| Tools across those servers collide on names, so the model cannot tell them apart by name alone | 775 tools carrying colliding names | Microsoft Research, 11 September 2025 |
| A 16,000-token model runs out of window once enough tools are described to it at once | 93 tools | Microsoft Research, 11 September 2025 |
| Cutting a default toolset improved task success and cut latency | 40 tools down to 13, worth 2 to 5 percentage points on SWE-bench Verified and SWE-Lancer with GPT-5 and Sonnet 4.5, and roughly 400ms | GitHub, 19 November 2025 |
| Selecting tools by relevance beat handing over a fixed list | 94.5% tool-use coverage against 69.0% | GitHub, 19 November 2025 |
| Loading tool descriptions on demand cut token use and raised tool-use accuracy | 85% fewer tokens; accuracy 49% to 74% on Opus 4 and 79.5% to 88.1% on Opus 4.5 | Anthropic, 24 November 2025 |
| Showing roughly seven tools matched the coverage of a fixed list of fifty | 90.3% against 90.8% | Meta preprint, 23 May 2026 |
| Choosing the tool list per question rather than fixing it raised tool-choice accuracy | 87.1% to 93.1% | Meta preprint, 23 May 2026 |

The results point the same way. Past a certain number, extra options stop paying for themselves: they cost window, they cost latency, and on some models they cost accuracy. Each team reached the same fix: work out which few actions this particular request needs, describe only those, and leave the rest out of the conversation.

That fix is cheap on a server, where you can afford a second model call to do the working out. On a phone, with a small window and a battery, you have to do it without one. That was the part of building a phone agent that took us longest.

## What that means for your work

Split the jobs you want done into the two categories above.

**Jobs a local model handles well on its own.** Draft the reply. Summarise the meeting. Tidy the note. Turn the voice memo into text. These stay on the phone, cost nothing per use, and work with no signal.

**Jobs that need to reach your accounts.** These are the four from earlier: the customer thread, the stock check, the payment link, the booking. Each needs an account connection you have granted, and a model chosen for the job rather than for the phone it runs on. They return hours where the first group returns minutes. That is the story in [how an AI agent runs a small business](/blog/ai-agent-runs-small-business).

Most people want both, which is why the sensible design leaves the choice with you. OpenAlly keeps the private text work on the device and lets you route the rest to a provider you already trust, a plan you already pay for, a server you run, or the OpenAlly service. The [trust and control guide](/under-the-hood) sets out where each route sends a request. "It runs locally" and "nothing ever leaves the phone" are different promises. The first is about where the model runs. The second is about every other request the app makes, and it is much harder to keep.

> Interface view: **Choose the route that fits this job.** An OpenAlly interface view of the current processing choices: on-device work, a provider key, a supported subscription, your own compatible server, or OpenAlly Cloud.

## Choosing what to run where

Match the job to the tool.

| What you want | What to use |
| --- | --- |
| Private drafting and summarising, no account, no connection | A dedicated local chat app such as PocketPal AI or MLC Chat, which is simpler than anything else here |
| The same, plus transcription and document work, inside a wider app | OpenAlly's on-device model tool, with the option to switch to a larger model when a job needs it |
| Anything that reads or changes a real account, order, invoice or booking | An agent with permissions you granted, such as OpenAlly, and a model chosen per job |
| Work that starts on a phone and continues on a laptop | A product that treats both as the same body of work |

> Interface view: **Start with the phone; continue with more room.** A source-aligned interface view of OpenAlly continuity: the same account, agents, and chat history are available on mobile and Mac, while Mac remains optional.

Write down whatever must keep working with no signal. That short list decides your local model. Everything else can afford a network request.

## Frequently asked questions

> Interface view: **Five checks before you choose an offline model.** The FAQ in one field card: platform support, working-memory headroom, privacy scope, tool reach, and the storage, battery and time costs of running locally.

### Does offline AI work on iPhone as well as Android?

Both platforms run local models. [PocketPal AI](https://github.com/a-ghorbani/pocketpal-ai) documents support for Android, iOS and iPadOS, and OpenAlly's [Local LLM tool](/features/tools/local-llm) is built for both. Where OpenAlly differs today is transcription: the [Whisper tool](/features/tools/whisper) is an Android capability. The [download page](/download) lists where OpenAlly stands on each platform.

### How much memory does my phone need?

More than the model file. Google's mobile-optimised [Gemma 3 1B is a 529MB download](https://developers.googleblog.com/google-ai-edge-small-language-models-multimodality-rag-function-calling/), and a running model needs working space on top of that beside the operating system and everything else open. In our own testing a 12GB handset had roughly 5.2GB genuinely free when a model tried to load. OpenAlly's device-memory floors of 6GiB and 8GiB are safety gates we chose, not hardware requirements anyone has published.

### Is a local model automatically private?

The model request is, because it has nowhere to go. An app around it can still send other things: analytics, crash reports, backups, or a fallback request to a service when the local model cannot handle a job. Read what the app says it sends. The word "local" does not tell you. OpenAlly's [privacy policy](/privacy) lists what leaves the device and when.

### Can a local model actually do things, or only chat?

Some can call tools, and the tools tend to be self-contained ones such as PocketPal's `calculate` and `datetime`. Reaching your messages, orders or calendar is a different layer that needs granted permissions and enough context space to describe the available actions. The published evidence, from [Microsoft Research](https://www.microsoft.com/en-us/research/blog/tool-space-interference-in-the-mcp-era-designing-for-agent-compatibility-at-scale/) through [Anthropic](https://www.anthropic.com/engineering/advanced-tool-use), [GitHub](https://github.blog/ai-and-ml/github-copilot/how-were-making-github-copilot-smarter-with-fewer-tools/) and [Meta's chance-corrected study](https://arxiv.org/abs/2605.24660), is that the answer is to show the model a small, relevant set of actions rather than all of them.

### Does running a model locally cost anything?

The download is one-off and the arithmetic is free after that. The cost shows up as storage, battery and speed. Our own [field test on a flagship handset](/blog/gemma-4-android-litert-lm) measured a first response in roughly 10 to 15 seconds. A cloud request over a decent connection usually starts answering in about a second. That is a fair trade for private work and a poor one when a customer is waiting.

## Declared sources

- [Google AI Edge: small language models with multimodality, RAG and function calling](https://developers.googleblog.com/google-ai-edge-small-language-models-multimodality-rag-function-calling/)
- [PocketPal AI repository](https://github.com/a-ghorbani/pocketpal-ai)
- [MLC LLM Android deployment documentation](https://llm.mlc.ai/docs/deploy/android.html)
- [Anthropic engineering: advanced tool use](https://www.anthropic.com/engineering/advanced-tool-use)
- [Microsoft Research: tool-space interference in the MCP era](https://www.microsoft.com/en-us/research/blog/tool-space-interference-in-the-mcp-era-designing-for-agent-compatibility-at-scale/)
- [GitHub: making Copilot smarter with fewer tools](https://github.blog/ai-and-ml/github-copilot/how-were-making-github-copilot-smarter-with-fewer-tools/)
- [How Many Tools Should an LLM Agent See? A Chance-Corrected Answer (Meta Platforms, preprint)](https://arxiv.org/abs/2605.24660)