Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract
summarize-extract-029
easy summarizationemail
The prompt
user
Summarize the neighborhood association notice below in at most 50 words. Reply with only the summary. """ Subject: Community pool closed Aug 3-14 for resurfacing Hello Willow Bend neighbors, The community pool will be closed from Monday, August 3 through Friday, August 14 while the contractor resurfaces the pool floor and replaces the cracked tiles along the steps. If the weather cooperates, the pool will reopen on Saturday, August 15 at 10 a.m. Heavy rain could push that date back a few days; we'll post updates on the clubhouse door and in the neighborhood app. During the closure, Willow Bend residents may use the pool at the Cedar Hollow clubhouse on Ridge Road. Bring your Willow Bend key fob - Cedar Hollow staff will check it at the gate. Guest passes are not valid there. The Stingrays swim team will move practices to the Lakeview Recreation Center for those two weeks. Coach Danielle will send practice times directly to swim families. Thank you for your patience - the new surface should last another 15 years. Willow Bend HOA Board """
What we expected
{
"kind": "summarize",
"word_limit": 50,
"facts": [
{
"id": "closed",
"desc": "pool closed August 3-14",
"any": [
[
"(close|closure|shut)",
"aug(ust|\\.)?\\s*3(rd)?\\b|\\b8/3\\b|\\b3(rd)?\\s*(-|to|through|until)\\s*(aug(ust|\\.)?\\s*)?(the\\s*)?14|\\b3(rd)?\\s+of\\s+aug"
]
]
},
{
"id": "why",
"desc": "for resurfacing / repairs",
"any": [
"resurfac|repair|tile|renovat"
]
},
{
"id": "reopen",
"desc": "reopens Saturday August 15 (weather permitting)",
"any": [
"(\\baug(ust|\\.)?\\s*15(st|nd|rd|th)?\\b|\\b15(st|nd|rd|th)?\\s+(of\\s+)?aug(ust|\\.)?\\b|\\b0?8/0?15\\b|\\b2026-08-15\\b|\\bfifteenth\\s+(of\\s+)?aug(ust|\\.)?\\b|\\baug(ust|\\.)?\\s+(the\\s+)?fifteenth\\b|\\bsat(urday|\\.)?\\b)"
]
},
{
"id": "alt_pool",
"desc": "residents can use Cedar Hollow pool with key fob",
"any": [
"cedar hollow"
]
},
{
"id": "swim",
"desc": "swim team practices move to Lakeview Rec Center",
"any": [
"lakeview|swim team|stingrays"
]
}
],
"allowed_numbers": [],
"source_words": 162,
"reference_summary": "The Willow Bend pool is closed August 3-14 for resurfacing and tile repairs, reopening Saturday, August 15 at 10 a.m. if weather allows. Meanwhile residents can use the Cedar Hollow pool with their key fob (no guests). Stingrays swim team practices move to the Lakeview Recreation Center."
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 2.51 s
The Willow Bend community pool closes Aug 3–14 for resurfacing, with an expected Aug 15 reopening at 10 a.m. Residents may use the Cedar Hollow clubhouse pool with a Willow Bend key fob; guest passes don't apply. Stingrays swim practices move to Lakeview Recreation Center.
Judge medians: Faithfulness 10 · Fluency and usefulness 10
Scoring detail
{
"facts": {
"closed": true,
"why": true,
"reopen": true,
"alt_pool": true,
"swim": true
},
"facts_frac": 1,
"word_count": 45,
"word_limit": 50,
"length_score": 1,
"unsupported_numbers": [],
"number_score": 1
}Ornith-1.5 9B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 2.00 s
The Willow Bend community pool closes Aug 3-14 for resurfacing and tile replacement, reopening Aug 15 at 10 a.m. (weather permitting). Residents can use the Cedar Hollow pool with their key fob. The Stingrays swim team moves practices to Lakeview Recreation Center.
Judge medians: Faithfulness 10 · Fluency and usefulness 10
Scoring detail
{
"facts": {
"closed": true,
"why": true,
"reopen": true,
"alt_pool": true,
"swim": true
},
"facts_frac": 1,
"word_count": 42,
"word_limit": 50,
"length_score": 1,
"unsupported_numbers": [],
"number_score": 1
}Granite 4.2 8B
✗ fail · score 94 (scripted checks 93, judges 9.5/10 from 3) · 2.56 s
The community pool will be closed August 3‑14 for resurfacing; if weather permits, it reopens Saturday August 15 at 10 a.m., otherwise may be delayed. Residents may use the Cedar Hollow clubhouse pool with their Willow Bend key fob; guest passes aren’t valid. Stingrays swim team practices move to Lakeview Recreation Center for two weeks.
Judge medians: Faithfulness 10 · Fluency and usefulness 9
Scoring detail
{
"facts": {
"closed": true,
"why": true,
"reopen": true,
"alt_pool": true,
"swim": true
},
"facts_frac": 1,
"word_count": 55,
"word_limit": 50,
"length_score": 0.5,
"unsupported_numbers": [],
"number_score": 1
}Spark-X2.5 4B
✓ pass · score 98 (scripted checks 100, judges 9.5/10 from 3) · 1.13 s
Willow Bend pool closed Aug 3-14 for resurfacing. Residents can use Cedar Hollow pool with key fob; Stingrays practice at Lakeview Rec Center. Reopens Aug 15 if weather permits.
Judge medians: Faithfulness 10 · Fluency and usefulness 9
Scoring detail
{
"facts": {
"closed": true,
"why": true,
"reopen": true,
"alt_pool": true,
"swim": true
},
"facts_frac": 1,
"word_count": 29,
"word_limit": 50,
"length_score": 1,
"unsupported_numbers": [],
"number_score": 1
}Mellum2.1 12B-A2.5B
✓ pass · score 98 (scripted checks 100, judges 9.5/10 from 3) · 0.97 s
Community pool closed Aug 3-14 for resurfacing; expected reopen Aug 15 at 10 a.m. if weather permits. Residents may use Cedar Hollow clubhouse pool with Willow Bend key fob; guest passes not accepted. Stingrays swim team practices moved to Lakeview Recreation Center.
Judge medians: Faithfulness 10 · Fluency and usefulness 9
Scoring detail
{
"facts": {
"closed": true,
"why": true,
"reopen": true,
"alt_pool": true,
"swim": true
},
"facts_frac": 1,
"word_count": 42,
"word_limit": 50,
"length_score": 1,
"unsupported_numbers": [],
"number_score": 1
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 83 (scripted checks 100, judges 6.5/10 from 3) · 0.60 s
The Willow Bend community pool will close August 3-14 for resurfacing and reopen Saturday, August 15 at 10 a.m. Residents can use the Cedar Hollow clubhouse instead. The Stingrays swim team will practice at Lakeview Recreation Center.
Judge medians: Faithfulness 6 · Fluency and usefulness 7
Scoring detail
{
"facts": {
"closed": true,
"why": true,
"reopen": true,
"alt_pool": true,
"swim": true
},
"facts_frac": 1,
"word_count": 37,
"word_limit": 50,
"length_score": 1,
"unsupported_numbers": [],
"number_score": 1
}MiniCPM5 2B
✗ fail · score 85 (scripted checks 80, judges 9.0/10 from 3) · 0.89 s
The community pool is closed Aug 3-14 for resurfacing. Reopen date may shift with weather. Residents can use the Cedar Hollow clubhouse, requiring a Willow Bend key fob. The Stingrays swim team moves practices to Lakeview Recreation Center.
Judge medians: Faithfulness 10 · Fluency and usefulness 8
Scoring detail
{
"facts": {
"closed": true,
"why": true,
"reopen": false,
"alt_pool": true,
"swim": true
},
"facts_frac": 0.8,
"word_count": 38,
"word_limit": 50,
"length_score": 1,
"unsupported_numbers": [],
"number_score": 1
}K2-Horizon 0.9B
✓ pass · score 93 (scripted checks 100, judges 8.5/10 from 3) · 1.14 s
Pool closed Aug 3‑14 for resurfacing; reopen Aug 15 10 am unless rain delays. Residents can use Cedar Hollow clubhouse on Ridge Road with key fob. Swim team moves to Lakeview Recreation Center. Board thanks.
Judge medians: Faithfulness 10 · Fluency and usefulness 7
Scoring detail
{
"facts": {
"closed": true,
"why": true,
"reopen": true,
"alt_pool": true,
"swim": true
},
"facts_frac": 1,
"word_count": 35,
"word_limit": 50,
"length_score": 1,
"unsupported_numbers": [],
"number_score": 1
}← summarize-extract-028All Summarize + extract itemssummarize-extract-030 →