Unified Memory

Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract

summarize-extract-031

medium summarizationarticle

The prompt

user

Summarize the news article below in at most 80 words. Reply with only the summary.

"""
Eastgate Library to close for 10-month renovation, reopen next spring

By Lorna Whitcombe, Brookhaven Ledger

The Eastgate branch of the Brookhaven Public Library will close on June 1 for a renovation that library officials say is long overdue. The $6.8 million project will gut the 1972 building's interior, replacing its heating and cooling system, adding a children's wing and a makerspace with 3D printers and sewing machines, and installing a new accessible entrance on the Maple Avenue side.

"Our patrons have been patient with leaky windows and an elevator that's out of service more often than not," said branch manager Felipe Ortega. "When this branch reopens, it's going to feel like a different building."

The work is scheduled to take about 10 months, with a reopening planned for spring 2027. Ortega cautioned that older buildings can hide surprises once walls are opened up, so the date could shift.

While the branch is closed, the library will run a temporary location in the former Pioneer Savings Bank building at 310 Grove Street, about four blocks away. The temporary site will offer holds pickup, a small browsing collection, public computers and printing, and weekly story time. It will be open Monday through Saturday, 10 a.m. to 6 p.m. Book returns can be made there or at any other branch.

Most of the project is paid for by the library bond voters approved in 2024. A $1.2 million gift from the Halvorsen Family Foundation covers the makerspace and children's wing. The library's board approved the construction contract with Beckett & Lowe Builders last week by a 6-1 vote. The lone no vote, trustee Grace Mendel, said she supported the renovation but objected to the contractor's timeline.

Meeting rooms will not be available at the temporary site. Community groups that regularly book space at Eastgate have been offered rooms at the Northside branch at no charge.

Residents can see renderings of the new building and sign up for construction updates on the library's website.
"""
What we expected
{
  "kind": "summarize",
  "word_limit": 80,
  "facts": [
    {
      "id": "closing",
      "desc": "Eastgate branch closes June 1",
      "any": [
        "(\\bjune?\\s*1(st|nd|rd|th)?\\b|\\b1(st|nd|rd|th)?\\s+(of\\s+)?june?\\b|\\b0?6/0?1\\b|\\b2026-06-01\\b|\\bfirst\\s+(of\\s+)?june?\\b|\\bjune?\\s+(the\\s+)?first\\b)"
      ]
    },
    {
      "id": "cost",
      "desc": "$6.8 million renovation",
      "any": [
        "6\\.8\\s*(million|m\\b)|6,800,000"
      ]
    },
    {
      "id": "reopen",
      "desc": "reopening planned for spring 2027 (about 10 months)",
      "any": [
        "spring|2027|10 months|ten months|ten-month|10-month"
      ]
    },
    {
      "id": "temp",
      "desc": "temporary location at former bank building, 310 Grove Street",
      "any": [
        "grove|pioneer|bank building|former bank"
      ]
    },
    {
      "id": "funding",
      "desc": "funded by 2024 bond plus $1.2M foundation gift",
      "any": [
        "bond|halvorsen|foundation|1\\.2\\s*(million|m\\b)|donation|gift"
      ]
    },
    {
      "id": "features",
      "desc": "new features: children's wing, makerspace, accessible entrance, HVAC",
      "any": [
        "children|makerspace|maker space|accessib|heating|hvac"
      ]
    }
  ],
  "allowed_numbers": [],
  "source_words": 330,
  "reference_summary": "Brookhaven's Eastgate library branch closes June 1 for a $6.8 million, roughly 10-month renovation and should reopen in spring 2027. The work adds a children's wing, a makerspace, new heating and cooling, and an accessible entrance. A temporary branch will run at 310 Grove Street in a former bank. Funding comes from the 2024 library bond plus a $1.2 million Halvorsen Family Foundation gift."
}

What each model answered

MiMo-V2.6 Distill 9B

✗ fail · score 82 (scripted checks 83, judges 8.0/10 from 3) · 3.55 s

The Brookhaven Public Library's Eastgate branch will close June 1 for a $6.8 million, 10-month renovation, reopening in spring 2027. The project adds a children's wing, makerspace, new HVAC and accessible entrance. A temporary site at 310 Grove Street will offer limited services; meeting rooms move to Northside. The Beckett & Lowe contract was approved 6-1.

Judge medians: Faithfulness 8 · Fluency and usefulness 8

Scoring detail
{
  "facts": {
    "closing": true,
    "cost": true,
    "reopen": true,
    "temp": true,
    "funding": false,
    "features": true
  },
  "facts_frac": 0.8333,
  "word_count": 55,
  "word_limit": 80,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

Ornith-1.5 9B

✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 3) · 3.65 s

The Brookhaven Public Library's Eastgate branch will close June 1 for a 10-month, $6.8 million renovation, reopening in spring 2027. The project will replace the heating and cooling system, add a children's wing and makerspace with 3D printers and sewing machines, and install a new accessible entrance. A temporary location at 310 Grove Street will offer holds pickup, computers, printing, and story time. Funding comes from 2024 bond voters and a $1.2 million Halvorsen Family Foundation gift.

Judge medians: Faithfulness 9 · Fluency and usefulness 9

Scoring detail
{
  "facts": {
    "closing": true,
    "cost": true,
    "reopen": true,
    "temp": true,
    "funding": true,
    "features": true
  },
  "facts_frac": 1,
  "word_count": 77,
  "word_limit": 80,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

Granite 4.2 8B

✗ fail · score 83 (scripted checks 85, judges 8.0/10 from 3) · 5.52 s

The Eastgate branch of Brookhaven Public Library will close June 1 for a 10‑month renovation costing $6.8 million, gutting the 1972 interior, upgrading heating/cooling, adding a children’s wing and makerspace with 3D printers and sewing machines, and installing a new accessible entrance. While closed, services will be provided from a temporary site in the former Pioneer Savings Bank building, offering holds pickup, browsing, computers, printing and weekly story time, Monday–Saturday 10 a.m.–6 p.m. Reopening is planned for spring 2027, though the date may shift due to unforeseen issues. Funding comes mainly from a 2024 library bond and a $1.2 million Halvorsen Family Foundation gift; meeting rooms will be unavailable at the temporary site, with community groups redirected to the Northside branch.

Judge medians: Faithfulness 10 · Fluency and usefulness 6

Scoring detail
{
  "facts": {
    "closing": true,
    "cost": true,
    "reopen": true,
    "temp": true,
    "funding": true,
    "features": true
  },
  "facts_frac": 1,
  "word_count": 121,
  "word_limit": 80,
  "length_score": 0,
  "unsupported_numbers": [],
  "number_score": 1
}

Spark-X2.5 4B

✗ fail · score 80 (scripted checks 100, judges 6.0/10 from 3) · 2.19 s

Eastgate Library will close June 1 for a $6.8 million renovation, reopening spring 2027. The 1972 building will be gutted, with new heating/cooling, a children's wing, makerspace, and accessible entrance. During closure, a temporary site at 310 Grove Street will offer services. The project is funded by bond voters, a family foundation, and a contractor approved by the board.

Judge medians: Faithfulness 5 · Fluency and usefulness 7

Scoring detail
{
  "facts": {
    "closing": true,
    "cost": true,
    "reopen": true,
    "temp": true,
    "funding": true,
    "features": true
  },
  "facts_frac": 1,
  "word_count": 59,
  "word_limit": 80,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

Mellum2.1 12B-A2.5B

✗ fail · score 87 (scripted checks 83, judges 9.0/10 from 3) · 1.30 s

Eastgate Library will close June 1 for a 10‑month renovation costing $6.8 million, including a new heating/cooling system, children’s wing, makerspace, and accessible entrance. A temporary site at 310 Grove Street will open during closure, offering limited services. Reopening planned for spring 2027; contract approved 6‑1.

Judge medians: Faithfulness 10 · Fluency and usefulness 8

Scoring detail
{
  "facts": {
    "closing": true,
    "cost": true,
    "reopen": true,
    "temp": true,
    "funding": false,
    "features": true
  },
  "facts_frac": 0.8333,
  "word_count": 46,
  "word_limit": 80,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

Ling-3.0-tiny (7.9B-A1.3B)

✗ fail · score 87 (scripted checks 83, judges 9.0/10 from 3) · 0.89 s

The Eastgate branch of the Brookhaven Public Library will close on June 1 for a $6.8 million renovation expected to take 10 months, with a reopening planned for spring 2027. The project includes a new accessible entrance, children's wing, and makerspace. A temporary location at the former Pioneer Savings Bank building will remain open during construction.

Judge medians: Faithfulness 10 · Fluency and usefulness 8

Scoring detail
{
  "facts": {
    "closing": true,
    "cost": true,
    "reopen": true,
    "temp": true,
    "funding": false,
    "features": true
  },
  "facts_frac": 0.8333,
  "word_count": 56,
  "word_limit": 80,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

MiniCPM5 2B

✗ fail · score 86 (scripted checks 93, judges 8.0/10 from 3) · 1.64 s

The Eastgate branch of Brookhaven Public Library will close for a 10-month renovation starting June 1, costing $6.8 million. The project, funded by library bond voters and a foundation gift, includes gutting the 1972 building's interior, adding a children's wing and makerspace, and installing new accessible features. Reopening is planned for spring 2027, though the date may shift. A temporary location at 310 Grove Street offers holds, computers, and story time Monday through Saturday. The library's board approved the construction contract, with one trustee voting no over the timeline.

Judge medians: Faithfulness 9 · Fluency and usefulness 7

Scoring detail
{
  "facts": {
    "closing": true,
    "cost": true,
    "reopen": true,
    "temp": true,
    "funding": true,
    "features": true
  },
  "facts_frac": 1,
  "word_count": 89,
  "word_limit": 80,
  "length_score": 0.5,
  "unsupported_numbers": [],
  "number_score": 1
}

K2-Horizon 0.9B

✓ pass · score 93 (scripted checks 100, judges 8.5/10 from 3) · 0.74 s

Eastgate Library will close June 1 for a $6.8 million renovation, gutting the 1972 building, installing a new accessible entrance, adding a children’s wing and makerspace, and replacing heating/cooling systems. The project is funded by bond voters and a $1.2 million foundation gift, with construction contract approved by a 6‑1 vote. The branch will reopen in spring 2027, while a temporary library site in the former Pioneer Savings Bank will operate until then.

Judge medians: Faithfulness 9 · Fluency and usefulness 8

Scoring detail
{
  "facts": {
    "closing": true,
    "cost": true,
    "reopen": true,
    "temp": true,
    "funding": true,
    "features": true
  },
  "facts_frac": 1,
  "word_count": 73,
  "word_limit": 80,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

← summarize-extract-030All Summarize + extract itemssummarize-extract-032 →