What is local AI?
Local AI is an AI model that runs on your own device. The model file is on your smartphone, tablet, or computer, and your device does the work. A cloud assistant is different. It sends your request to a data centre, and the answer comes back over the internet.
What does local AI give you?
- Privacy. When you use the model, your requests and its answers stay on your device.
- No internet. The model works on a plane and in places with a weak signal.
- No fee per message. You do not pay a provider each time you use the model.
- Control. A model file that you keep does not change unless you change it.
Why has local AI been hard?
The strongest models need a data centre, not a phone. A big model on a laptop is slow, and you wait for each word. Small models run fast, but they get hard things wrong. So local AI has been too big, too slow, or not good enough.
How do small models get better?
Two methods help. Optimization makes a model file smaller and faster, with as little loss of quality as possible. Training on examples makes a model better at the work that you give it.
In one of our tests, a small model trained on examples scored 88.1 percent on a data privacy task. Claude Opus 4.5 scored 70.7 percent.
Added after the results.
Sample of 300 synthetic documents.
The benchmarks guide gives the wins and the losses.
How fast is local AI?
Speed is measured in tokens per second. A token is a small piece of a word. On a computer, our trained model writes up to 75 tokens per second and reads up to 1,263 tokens per second. That is faster than you can read.
Where do big cloud models still lead?
Big cloud models lead most on long tasks with many steps.
On many-turn tasks we score 15.0 and Claude Opus 4.5 scores 81.0.
So use a local model for work that you do often, and a big cloud model for a long task with many steps.
What do you need to run local AI?
You need a device with enough memory and a model file that fits in that memory. Memory is the main limit, because the model stays in memory while it runs. Read the guides for Mac computers and for smartphones.
Ollama, LM Studio, and PocketPal run models that other people made, and they work today. Third Weather does a different thing. It will make a model from your own examples. Read how to train AI on your own work.
Speeds come from our trained Qwen3.5-0.8B model file qwen3.5-0.8b-Q4_K_M.gguf (Q4_K_M) on one tested computer with 16 GB. Reads and writes come from llama-bench on the GPU, the mean of 3 runs, a short diagnostic, not our ten-minute test.