Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread
rewrite-033
easy shortennotice
The prompt
user
Shorten the text below to at most 40 words. Keep the key facts (names, dates, times, amounts, numbers) accurate, and do not add anything new. Reply with only the shortened text, with no preamble and no word count. """ The Maple Grove Branch Library will be closed from Monday, July 7 through Sunday, July 13 while we replace the carpet and repaint the reading room. During the closure, you can return books in the outdoor drop box, which will be emptied every morning. Holds that are ready for pickup will be held until July 20, so you will not lose your place in line. The Oak Street Branch will remain open regular hours if you need to borrow materials during the week. We apologize for the inconvenience and look forward to welcoming you back to a refreshed space. """
What we expected
{
"type": "shorten",
"source": "The Maple Grove Branch Library will be closed from Monday, July 7 through Sunday, July 13 while we replace the carpet and repaint the reading room. During the closure, you can return books in the outdoor drop box, which will be emptied every morning. Holds that are ready for pickup will be held until July 20, so you will not lose your place in line. The Oak Street Branch will remain open regular hours if you need to borrow materials during the week. We apologize for the inconvenience and look forward to welcoming you back to a refreshed space.",
"max_words": 40,
"facts": [
{
"name": "Maple Grove",
"any": [
"maple grove"
]
},
{
"name": "July 7",
"any": [
"july 7",
"jul 7",
"jul. 7"
]
},
{
"name": "July 13",
"any": [
"july 13",
"jul 13",
"jul. 13"
],
"re": "jul(?:y|\\.)? 7(?:th)?\\s*(?:-|to|through|thru|until)\\s*(?:jul(?:y|\\.)? )?13"
},
{
"name": "drop box",
"any": [
"drop box",
"dropbox",
"drop-box",
"book drop"
]
},
{
"name": "holds until July 20",
"any": [
"july 20"
]
},
{
"name": "Oak Street open",
"any": [
"oak street"
]
}
],
"banned": [],
"banned_re": [],
"extra_numbers_ok": [],
"reference": "Maple Grove Branch Library will be closed July 7 through July 13 for carpet and painting. Return books to the outdoor drop box. Ready holds will be kept until July 20. Oak Street Branch stays open regular hours."
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 97 (scripted checks 100, judges 9.3/10 from 3) · 1.86 s
Maple Grove Branch closes July 7–13 for carpet replacement and reading room repainting. Return books in the outdoor drop box, emptied daily. Ready holds stay until July 20. Oak Street remains open.
Judge medians: Faithfulness 10 · Key information kept 9 · Readability 9
Scoring detail
{
"cleaned_words": 32,
"similarity_to_source": 0.305,
"facts_missing": [],
"new_numbers": [],
"max_words": 40,
"gate": 1,
"checks": {
"facts": 1,
"numbers": 1,
"length": 1
}
}Ornith-1.5 9B
✗ fail · score 90 (scripted checks 83, judges 9.7/10 from 3) · 1.81 s
The Maple Grove Branch Library will be closed July 7–13 for carpet replacement and reading room repainting. Return books in the outdoor drop box, emptied daily. Holds stay ready until July 20. Oak Street Branch stays open regular hours. We apologize for the inconvenience.
Judge medians: Faithfulness 10 · Key information kept 10 · Readability 9
Scoring detail
{
"cleaned_words": 44,
"similarity_to_source": 0.462,
"facts_missing": [],
"new_numbers": [],
"max_words": 40,
"gate": 1,
"checks": {
"facts": 1,
"numbers": 1,
"length": 0.5
}
}Granite 4.2 8B
✓ pass · score 98 (scripted checks 100, judges 9.7/10 from 3) · 1.78 s
Maple Grove Branch Library closed July 7–13 for carpet replacement and reading‑room repaint; use outdoor drop box for returns (emptied each morning). Holds ready for pickup will be held until July 20. Oak Street Branch remains open regular hours.
Judge medians: Faithfulness 10 · Key information kept 10 · Readability 9
Scoring detail
{
"cleaned_words": 39,
"similarity_to_source": 0.348,
"facts_missing": [],
"new_numbers": [],
"max_words": 40,
"gate": 1,
"checks": {
"facts": 1,
"numbers": 1,
"length": 1
}
}Spark-X2.5 4B
✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 3) · 1.08 s
The Maple Grove Branch Library is closed July 7-13 for carpet and repaint. Return books at the outdoor drop box, emptied daily. Holds remain until July 20. The Oak Street Branch stays open.
Judge medians: Faithfulness 10 · Key information kept 9 · Readability 8
Scoring detail
{
"cleaned_words": 33,
"similarity_to_source": 0.348,
"facts_missing": [],
"new_numbers": [],
"max_words": 40,
"gate": 1,
"checks": {
"facts": 1,
"numbers": 1,
"length": 1
}
}Mellum2.1 12B-A2.5B
✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 3) · 0.82 s
Maple Grove Branch Library closed July 7‑13 for carpet replacement and reading room repaint. Return books via outdoor drop box (emptied daily). Holds ready for pickup held until July 20. Oak Street Branch remains open. Apologize for inconvenience.
Judge medians: Faithfulness 10 · Key information kept 10 · Readability 7
Scoring detail
{
"cleaned_words": 38,
"similarity_to_source": 0.365,
"facts_missing": [],
"new_numbers": [],
"max_words": 40,
"gate": 1,
"checks": {
"facts": 1,
"numbers": 1,
"length": 1
}
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 88 (scripted checks 83, judges 9.3/10 from 3) · 0.62 s
The Maple Grove Branch Library will be closed July 7–13 for carpet replacement and repainting. The Oak Street Branch will remain open. Book returns are available at the outdoor drop box, emptied daily. Holds will be held until July 20. We apologize for the inconvenience.
Judge medians: Faithfulness 10 · Key information kept 9 · Readability 9
Scoring detail
{
"cleaned_words": 45,
"similarity_to_source": 0.306,
"facts_missing": [],
"new_numbers": [],
"max_words": 40,
"gate": 1,
"checks": {
"facts": 1,
"numbers": 1,
"length": 0.5
}
}MiniCPM5 2B
✗ fail · score 61 (scripted checks 33, judges 9.0/10 from 3) · 1.12 s
The Maple Grove Branch Library will close from July 7 to July 13 while we replace carpet and repaint the reading room. You can return books in the outdoor drop box, emptied daily. Holds ready for pickup will be held until July 20. The Oak Street Branch will remain open regular hours. We apologize for the inconvenience.
Judge medians: Faithfulness 9 · Key information kept 9 · Readability 9
Scoring detail
{
"cleaned_words": 57,
"similarity_to_source": 0.654,
"facts_missing": [],
"new_numbers": [],
"max_words": 40,
"gate": 0.5,
"checks": {
"facts": 1,
"numbers": 1,
"length": 0
}
}K2-Horizon 0.9B
✗ fail · score 74 (scripted checks 85, judges 6.3/10 from 3) · 0.23 s
The Maple Grove Branch Library closes July 7-13 for carpet and repaint. Outdoor drop boxes stay open. Oak Street Branch stays open.
Judge medians: Faithfulness 8 · Key information kept 5 · Readability 6
Scoring detail
{
"cleaned_words": 22,
"similarity_to_source": 0.215,
"facts_missing": [
"drop box",
"holds until July 20"
],
"new_numbers": [],
"max_words": 40,
"gate": 1,
"checks": {
"facts": 0.6667,
"numbers": 1,
"length": 1
}
}