Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Read images
vision-007
easy screenshot
The prompt
1 image attached (not shown).
user
I don't want to install this update right now, but I want to be asked again later. Which button should I press? Reply with only the button's label.
What we expected
{
"answer": "Remind Me Tomorrow",
"accept": [],
"type": "choice",
"options": [
[
"Remind Me Tomorrow"
],
[
"Install Now"
],
[
"View Details"
]
]
}What each model answered
LFM2.5-VL 3B
✓ pass · score 100 · 0.52 s
Remind Me Tomorrow
Scoring detail
{
"type": "choice",
"normalized": "remind me tomorrow"
}MiMo-V2.6 Distill 9B
✓ pass · score 100 · 1.29 s
Remind Me Tomorrow
Scoring detail
{
"type": "choice",
"normalized": "remind me tomorrow"
}MiniCPM-V 4.7 1B
✓ pass · score 100 · 0.72 s
Remind Me Tomorrow
Scoring detail
{
"type": "choice",
"normalized": "remind me tomorrow"
}