Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Read images
vision-009
medium screenshot
The prompt
1 image attached (not shown).
user
According to this thermostat app, what is the current indoor temperature in degrees? Reply with only the number (no units or currency symbols).
What we expected
{
"answer": 66,
"accept": [],
"type": "number"
}What each model answered
LFM2.5-VL 3B
✓ pass · score 100 · 2.76 s
66
Scoring detail
{
"type": "number",
"parsed": 66
}MiMo-V2.6 Distill 9B
✓ pass · score 100 · 1.85 s
66
Scoring detail
{
"type": "number",
"parsed": 66
}MiniCPM-V 4.7 1B
✓ pass · score 100 · 1.21 s
66
Scoring detail
{
"type": "number",
"parsed": 66
}