Run local AI on a Mac
You can run local AI on a Mac. The model runs on the computer itself, so your words stay on it, and the model works with no internet. Mac computers with Apple silicon are a good fit, because the processor and the graphics processor share one pool of memory.
How much memory do you need?
Memory is the main limit. The model must fit in memory, together with macOS and your other apps. A model runs well when it fits with room to spare. It slows down when the computer must move memory to the disk.
- With 8 GB of memory, use a small model, and close heavy apps before you start.
- With 16 GB of memory, you can use a bigger model and keep your apps open.
What is a 4-bit model file?
A model file holds a very large set of numbers. A 4-bit file stores each number in fewer bits, so the file uses less memory and runs faster. The cost is some loss of quality. The technical name is quantization.
How fast is local AI on a Mac?
On a Mac, our trained model writes up to 75 tokens per second and reads up to 1,263 tokens per second. A token is a small piece of a word. That is faster than you can read.
How do you run a model on a Mac today?
Ollama and LM Studio run open models on a Mac. MLX is an open framework from Apple for machine learning on Apple silicon, and it can also train a model. To get a model that works the way you work, you can train it on your own examples. Read how to train AI on your own work.
When can you use our models on a Mac?
Our early models come to macOS computers. Alpha testing is running now.
We will tell you when our models are ready.
Speeds come from our trained Qwen3.5-0.8B model file qwen3.5-0.8b-Q4_K_M.gguf (Q4_K_M) on one tested computer with 16 GB. Reads and writes come from llama-bench on the GPU, the mean of 3 runs, a short diagnostic, not our ten-minute test.