Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Read images
vision-006
medium screenshot
The prompt
1 image attached (not shown).
user
I want to close this spreadsheet and throw away my edits. Which button should I press? Reply with only the button's label.
What we expected
{
"answer": "Don't Save",
"accept": [
"Dont Save",
"Do not save"
],
"type": "choice",
"options": [
[
"Don't Save",
"Dont Save",
"Do not save"
],
[
"Cancel"
],
[
"Save"
]
]
}What each model answered
LFM2.5-VL 3B
✗ fail · score 0 · 0.47 s
Cancel
Scoring detail
{
"type": "choice",
"normalized": "cancel",
"options_named": [
1
]
}MiMo-V2.6 Distill 9B
✓ pass · score 100 · 1.06 s
Don't Save
Scoring detail
{
"type": "choice",
"normalized": "dont save"
}MiniCPM-V 4.7 1B
✗ fail · score 0 · 0.72 s
Cancel
Scoring detail
{
"type": "choice",
"normalized": "cancel",
"options_named": [
1
]
}