Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Read images
vision-011
medium signphoto
The prompt
1 image attached (not shown).
user
What is the street name on this sign? Reply with only the street name as written (include the direction and street type).
What we expected
{
"answer": "W Alder Crest Ave",
"accept": [
"West Alder Crest Avenue",
"W Alder Crest Avenue",
"West Alder Crest Ave",
"Alder Crest Ave",
"Alder Crest Avenue"
],
"type": "text"
}What each model answered
LFM2.5-VL 3B
✓ pass · score 100 · 0.51 s
W Alder Crest AVE
Scoring detail
{
"type": "text",
"normalized": "w alder crest ave"
}MiMo-V2.6 Distill 9B
✓ pass · score 100 · 1.41 s
W Alder Crest Ave
Scoring detail
{
"type": "text",
"normalized": "w alder crest ave"
}MiniCPM-V 4.7 1B
✓ pass · score 100 · 1.27 s
W Alder Crest AVE
Scoring detail
{
"type": "text",
"normalized": "w alder crest ave"
}