Unified MemoryBenchmarks / Oct 21, 2026
Which Small AI Model Should You Run on a Mac? We Ran 312 Tests
A 2B model came 3rd of 8 at being a home assistant, ahead of an 8B and a 12B. A 3B read images better than a 9B. So which small AI model should you actually run on your Mac?
Read the full analysis →
A job-by-job look at how eight small text models and three image readers did on 312 everyday tests on one M4 Max, with the speed and memory each one costs.
Tested on Apple M4 Max (32-core GPU, 36 GB), macOS 27.0.1. Results generated Oct 21, 2026.
Overall ranking (text jobs)
Quality is the mean of the per-job accuracies (0-100) over the 6 text jobs, each job weighted equally. Speed and memory are shown beside it, never blended in.
| # | Model | Quality | Pass rate | Tok/s | Time, all jobs | RAM in use | RAM, 8K context full |
|---|---|---|---|---|---|---|---|
| 1 | MiMo-V2.6 Distill 9B | 91.0 | 80.5% | 37.8 | 12m 12s | 7.82 GB | 8.03 GB |
| 2 | Ornith-1.5 9B | 88.4 | 76.8% | 46.1 | 10m 15s | 5.52 GB | 6.44 GB |
| 3 | Granite 4.2 8B | 84.4 | 65.5% | 42.6 | 15m 08s | 5.54 GB | 8.13 GB |
| 4 | Spark-X2.5 4B | 83.1 | 65.8% | 65.6 | 6m 35s | 3.56 GB | 3.79 GB |
| 5 | Mellum2.1 12B-A2.5B | 81.9 | 60.7% | 101.2 | 5m 56s | 8.15 GB | 8.34 GB |
| 6 | Ling-3.0-tiny (7.9B-A1.3B) | 80.5 | 61.0% | 117.9 | 3m 17s | 5.07 GB | 5.17 GB |
| 7 | MiniCPM5 2B | 77.8 | 55.5% | 99.3 | 4m 05s | 1.64 GB | 2.36 GB |
| 8 | K2-Horizon 0.9B | 55.9 | 28.3% | 218.4 | 2m 20s | 1.24 GB | 1.33 GB |
Quality against what it costs you
Every model on every job
Accuracy per job (0-100). Brighter cyan is better; the scale runs from 40 (surface colour) to 100 (full cyan). Click a job for its items, or a model for its breakdown.
Reading images
Only models that accept images run the vision job, so it is ranked separately.
The jobs
tools · 50 items
Assistant actions
Turn everyday home/phone requests into the right tool calls, or ask / just answer when no tool fits.
rag · 40 items
Ask my documents
Answer questions about a folder of 22 household documents, with citations, and refuse when the answer isn't there.
chat · 42 items
Rewrite and proofread
Fix grammar and spelling without rewording, change a message's tone without losing facts, and shorten text to a word limit.
chat · 44 items
Summarize + extract
Pull exact JSON out of messy receipts, invoices, invites, shipping emails and signatures, and summarize emails, transcripts and articles.
chat · 40 items
Translate
Translate everyday texts (signs, chats, emails, reminders, recipes, product specs) between English and Spanish, French, German, Japanese, Chinese and Brazilian Portuguese.
chat · 56 items
Sort and triage
Label inbox emails, route support tickets by category and priority, and read review sentiment.
vision · 40 items
Read images
Read receipts, screenshots, signs, labels, charts, calendars and handwriting from a photo or screenshot, and answer with the exact value.
Method
= mean item score x 100 (failed requests count as 0). Pass rate = share of items passed. Overall quality = weighted mean of job accuracies over the text jobs (chat/tools/rag), each job weight 1 unless its job.json sets "weight"; item counts don't matter. Vision jobs are ranked separately (only vision models run them). Speed and memory are reported next to quality, never blended into it. A model is ranked only when it has a valid result for every item of every text job.
How memory was measured
context tokens: 8192
context why: Same llama-server context (-c) for every model; the largest request in the job sets is ~2,900 prompt tokens (a vision item) + 1,024 max_tokens = ~3,900, so 8,192 fits every item. mlx_lm.server has no context flag (KV cache grows on demand).
server: As um-jobs run starts it (llama-server -np 1 --load-mode none -fa on --jinja; mlx_lm.server), context overridden; one server at a time under the machine lock.
loaded: Server process-tree phys_footprint once ready (after the 4-token readiness request), median of 6 samples over 3 s. llama-server: weights + KV cache reserved for the full context + compute buffers. mlx_lm.server: weights.
in use: Same, after one warm-up request: summarize-extract/summarize-extract-033 transcript with a long-write-up instruction (~1,400 prompt tokens), max_tokens 200, temperature 0, lineup chat_template_kwargs.
full context: Same, after a second request that fills the context: summarize-extract documents summarize-extract-033, summarize-extract-034, summarize-extract-042, summarize-extract-038, summarize-extract-031, summarize-extract-030, summarize-extract-036, summarize-extract-044, summarize-extract-040, summarize-extract-013, summarize-extract-032 back to back (~7,000 prompt tokens), max_tokens 200. mlx_lm.server then holds an ~8K cache too, so this is the like-for-like figure.
disk: Model files in the Hugging Face cache: GGUF weights (+ mmproj for vision), or MLX safetensors.
generated: 2026-10-21T08:34:52+00:00
Video chapters
- 00:00 A 2B model beat a 12B?
- 00:22 The book club email test
- 01:05 The lineup: 8 text + 3 vision models
- 01:52 How we scored them
- 02:33 Job 1: Assistant actions
- 03:36 Job 2: Ask my documents
- 04:33 Job 3: Rewrite and proofread
- 05:16 Job 4: Summarize and extract
- 05:57 Job 5: Translation
- 06:42 Job 6: Sort and triage
- 07:31 Job 7: Reading images
- 08:20 Overall standings
- 08:49 Speed and memory
- 09:26 Why Ling is so fast: mixture of experts
- 10:02 Pick: best overall
- 10:28 Picks: best small, fastest, images
- 11:10 The fine print