Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Read images
vision-023
easy chart
The prompt
1 image attached (not shown).
user
Which month had the highest sales in this chart? Reply with only the month as written on the chart.
What we expected
{
"answer": "Apr",
"accept": [
"April"
],
"type": "choice",
"options": [
[
"Jan",
"January"
],
[
"Feb",
"February"
],
[
"Mar",
"March"
],
[
"Apr",
"April"
],
[
"May"
],
[
"Jun",
"June"
]
]
}What each model answered
LFM2.5-VL 3B
✓ pass · score 100 · 2.95 s
Apr
Scoring detail
{
"type": "choice",
"normalized": "apr"
}MiMo-V2.6 Distill 9B
✓ pass · score 100 · 1.93 s
Apr
Scoring detail
{
"type": "choice",
"normalized": "apr"
}MiniCPM-V 4.7 1B
✓ pass · score 100 · 1.15 s
Apr
Scoring detail
{
"type": "choice",
"normalized": "apr"
}