Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread
rewrite-038
medium shortennotes
The prompt
user
Shorten the text below to at most 50 words. Keep the key facts (names, dates, times, amounts, numbers) accurate, and do not add anything new. Reply with only the shortened text, with no preamble and no word count. """ Okay so for the community garden meeting on Saturday, here is where we landed after a pretty long discussion. First, plot fees are going up from $35 to $40 for the season starting in March, mainly because the water bill went up last year. Second, we agreed that every plot holder needs to do at least four hours of shared work, like weeding paths or turning the compost, before the end of August. There will be a sign-up sheet on the shed door. Third, Gwen volunteered to coordinate the tool library, so if you borrow anything, check with her. Finally, the next meeting is April 5 at 10 a.m. at the shed. """
What we expected
{
"type": "shorten",
"source": "Okay so for the community garden meeting on Saturday, here is where we landed after a pretty long discussion. First, plot fees are going up from $35 to $40 for the season starting in March, mainly because the water bill went up last year. Second, we agreed that every plot holder needs to do at least four hours of shared work, like weeding paths or turning the compost, before the end of August. There will be a sign-up sheet on the shed door. Third, Gwen volunteered to coordinate the tool library, so if you borrow anything, check with her. Finally, the next meeting is April 5 at 10 a.m. at the shed.",
"max_words": 50,
"facts": [
{
"name": "$35 to $40",
"any": [
"$35"
]
},
{
"name": "$40",
"any": [
"$40"
]
},
{
"name": "March",
"any": [
"march"
]
},
{
"name": "four hours",
"any": [
"four hours",
"4 hours"
]
},
{
"name": "August",
"any": [
"august"
]
},
{
"name": "sign-up sheet",
"any": [
"sign-up",
"sign up",
"signup"
]
},
{
"name": "Gwen tools",
"any": [
"gwen"
]
},
{
"name": "tool library",
"any": [
"tool"
]
},
{
"name": "April 5",
"any": [
"april 5"
]
},
{
"name": "10 am",
"any": [
"10 a.m",
"10 am",
"10am",
"10:00"
]
}
],
"banned": [],
"banned_re": [],
"extra_numbers_ok": [
"4"
],
"reference": "Garden plot fees rise from $35 to $40 starting in March due to higher water costs. Each plot holder must do four hours of shared work by the end of August (sign-up sheet on the shed door). Gwen runs the tool library. Next meeting: April 5, 10 a.m., shed."
}What each model answered
MiMo-V2.6 Distill 9B
✗ fail · score 83 (scripted checks 83, judges 8.3/10 from 2) · 2.48 s
After Saturday's discussion: plot fees rise from $35 to $40 for the March season due to higher water bills. Each plot holder must complete four hours of shared work before August's end; sign-up is on the shed door. Gwen will coordinate the tool library. Next meeting: April 5 at 10 a.m. at the shed.
Judge medians: Faithfulness 9 · Key information kept 7.5 · Readability 8.5
Scoring detail
{
"cleaned_words": 54,
"similarity_to_source": 0.434,
"facts_missing": [],
"new_numbers": [],
"max_words": 50,
"gate": 1,
"checks": {
"facts": 1,
"numbers": 1,
"length": 0.5
}
}Ornith-1.5 9B
✗ fail · score 65 (scripted checks 33, judges 9.7/10 from 3) · 2.95 s
At Saturday's community garden meeting, plot fees rise from $35 to $40 for the season starting in March, due to a higher water bill. Each plot holder must complete at least four hours of shared work, like weeding paths or turning compost, before the end of August. A sign-up sheet will be on the shed door. Gwen will coordinate the tool library, so check with her to borrow anything. The next meeting is April 5 at 10 a.m. at the shed.
Judge medians: Faithfulness 10 · Key information kept 10 · Readability 9
Scoring detail
{
"cleaned_words": 81,
"similarity_to_source": 0.642,
"facts_missing": [],
"new_numbers": [],
"max_words": 50,
"gate": 0.5,
"checks": {
"facts": 1,
"numbers": 1,
"length": 0
}
}Granite 4.2 8B
✗ fail · score 51 (scripted checks 23, judges 8.0/10 from 3) · 2.68 s
For the Saturday community garden meeting, plot fees rise from $35 to $40 starting March, due to last year's higher water bill. Plot holders must complete at least four hours of shared work (weeding paths or compost turning) by August 31. A sign‑up sheet will be on the shed door. Gwen volunteers to coordinate the tool library. The next meeting is April 5 at 10 a.m. at the shed.
Judge medians: Faithfulness 8 · Key information kept 8 · Readability 8
Scoring detail
{
"cleaned_words": 69,
"similarity_to_source": 0.508,
"facts_missing": [],
"new_numbers": [
"31"
],
"max_words": 50,
"gate": 0.5,
"checks": {
"facts": 1,
"numbers": 0,
"length": 0
}
}Spark-X2.5 4B
✗ fail · score 83 (scripted checks 96, judges 7.0/10 from 3) · 1.35 s
For the community garden meeting Saturday, plot fees rise from $35 to $40 starting March due to higher water bills. Plot holders must do four hours of shared work by August. Gwen coordinates the tool library. The next meeting is April 5 at 10 a.m. at the shed.
Judge medians: Faithfulness 7 · Key information kept 6 · Readability 8
Scoring detail
{
"cleaned_words": 48,
"similarity_to_source": 0.45,
"facts_missing": [
"sign-up sheet"
],
"new_numbers": [],
"max_words": 50,
"gate": 1,
"checks": {
"facts": 0.9,
"numbers": 1,
"length": 1
}
}Mellum2.1 12B-A2.5B
✗ fail · score 80 (scripted checks 83, judges 7.7/10 from 3) · 1.00 s
Plot fees rise from $35 to $40 starting March due to higher water costs. Plot holders must complete at least four hours of shared work (e.g., weeding, compost turning) by August; sign‑up sheet on shed door. Gwen will coordinate the tool library. Next meeting: April 5 at 10 a.m. at the shed.
Judge medians: Faithfulness 7 · Key information kept 7 · Readability 9
Scoring detail
{
"cleaned_words": 52,
"similarity_to_source": 0.402,
"facts_missing": [],
"new_numbers": [],
"max_words": 50,
"gate": 1,
"checks": {
"facts": 1,
"numbers": 1,
"length": 0.5
}
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 78 (scripted checks 96, judges 6.0/10 from 3) · 0.65 s
For the community garden meeting on Saturday, plot fees rise from $35 to $40 starting March. Every plot holder must do four hours of shared work before August. Gwen will coordinate the tool library. The next meeting is April 5 at 10 a.m. at the shed.
Judge medians: Faithfulness 5 · Key information kept 5 · Readability 8
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.506,
"facts_missing": [
"sign-up sheet"
],
"new_numbers": [],
"max_words": 50,
"gate": 1,
"checks": {
"facts": 0.9,
"numbers": 1,
"length": 1
}
}MiniCPM5 2B
✓ pass · score 87 (scripted checks 100, judges 7.3/10 from 3) · 0.87 s
Community garden meeting: plot fees rise from $35 to $40 for March. Each holder must do four hours of shared work by August. Sign-up sheet at the shed. Gwen will coordinate the tool library. Next meeting: April 5 at 10 a.m. at the shed.
Judge medians: Faithfulness 6 · Key information kept 7 · Readability 9
Scoring detail
{
"cleaned_words": 44,
"similarity_to_source": 0.41,
"facts_missing": [],
"new_numbers": [],
"max_words": 50,
"gate": 1,
"checks": {
"facts": 1,
"numbers": 1,
"length": 1
}
}K2-Horizon 0.9B
✗ fail · score 72 (scripted checks 91, judges 5.3/10 from 3) · 0.29 s
Plot fees up to $40. Every holder must do at least four hours of shared work by August. Sign-up sheet on shed door. Gwen coordinates tool library. Next meeting April 5, 10 a.m.
Judge medians: Faithfulness 5 · Key information kept 5 · Readability 6
Scoring detail
{
"cleaned_words": 33,
"similarity_to_source": 0.359,
"facts_missing": [
"$35 to $40",
"March"
],
"new_numbers": [],
"max_words": 50,
"gate": 1,
"checks": {
"facts": 0.8,
"numbers": 1,
"length": 1
}
}