Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Models
Ornith-1.5 9B
ornith-ai/Ornith-1.5-9B-MLX-4bit · MLX · mlx_lm.server 0.32.2
#2 of 8
overall, text jobs
88.4
quality score
46.1
median tokens/s
6.44 GB
RAM with 8K context filled
4.69 GB
download size
Score per job
| Job | Accuracy | Pass | Easy | Medium | Hard | Median latency | Tok/s | Job time | Errors |
|---|---|---|---|---|---|---|---|---|---|
| Assistant actions | 86.0 | 82.0% | 93.3% | 88.2% | 66.7% | 1.63 s | 41.0 | 1m 31s | 0 |
| Ask my documents | 78.9 | 77.5% | 91.7% | 92.3% | 53.3% | 3.33 s | 47.5 | 2m 20s | 0 |
| Rewrite and proofread | 88.2 | 59.5% | 71.4% | 71.4% | 35.7% | 2.61 s | 45.4 | 1m 58s | 0 |
| Summarize + extract | 89.9 | 59.1% | 66.7% | 76.9% | 37.5% | 3.27 s | 46.1 | 2m 36s | 0 |
| Translate | 91.9 | 82.5% | 100.0% | 84.6% | 64.3% | 1.65 s | 45.8 | 1m 04s | 0 |
| Sort and triage | 95.5 | 94.6% | 100.0% | 94.7% | 90.0% | 0.67 s | 47.3 | 47.2 s | 0 |
Easy / medium / hard columns are pass rates within each difficulty tier.
Memory and loading
| Download size | 4.69 GB |
| Loaded, idle | 5.40 GB |
| In use (8K context) | 5.52 GB |
| 8K context filled | 6.44 GB |
| Server peak during the jobs (incl. reserved context) | 8.24 GB |
| Load time | 6.00 s |
Measured on Apple M4 Max, 36 GB, macOS 27.0.1. How memory was measured.