Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract
summarize-extract-026
hard extractionemail
The prompt
user
Extract information from the meeting notes below. Reply with only a JSON object with exactly these keys:
- "meeting_date" (string "YYYY-MM-DD"): date of the meeting
- "attendees" (array): array of people who were present, each {"name": string, "email": string or null, "phone": string or null}
- "buildout_estimate" (number): build-out cost estimate
- "next_meeting_date" (string "YYYY-MM-DD"): date of the next meeting
Rules: include every key listed. Use null for any field that is not stated in the text (do not guess). Dates are YYYY-MM-DD. Times are HH:MM in 24-hour format. Amounts and counts are plain JSON numbers (no currency symbols, no commas). Phone numbers are in international format with + and the country code, for example +1 312 555 0100; leave out any national trunk prefix 0, so (0)20 or 020 becomes 20 after +44.
Meeting notes:
"""
Millbrook Food Co-op - Expansion Committee
Notes from Wednesday, October 14, 2026 (taken by Rosa)
Present: Rosa Delgado (rosa.delgado@millbrookcoop.org), Tom Hirsch (cell 919-555-0127), Anjali Mehta - anjali.mehta@millbrookcoop.org / 919.555.0184, and Keith from facilities (doesn't have a co-op email yet). Bea Lindgren sent regrets.
1. Lease for the Oak St storefront: landlord wants a 5-year term; we'd prefer 3. Tom to counter.
2. Budget: build-out estimate is $48,500; reserve fund is $61,200.
3. Keith to get two HVAC quotes.
We'll meet again two weeks from today, same time.
"""What we expected
{
"kind": "extract",
"answer": {
"meeting_date": "2026-10-14",
"attendees": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"buildout_estimate": 48500,
"next_meeting_date": "2026-10-28"
},
"fields": {
"meeting_date": {
"type": "date"
},
"attendees": {
"type": "list",
"item_fields": {
"name": {
"type": "text",
"needles_map": {
"Rosa Delgado": [
"rosa"
],
"Tom Hirsch": [
"tom",
"hirsch"
],
"Anjali Mehta": [
"anjali"
],
"Keith": [
"keith"
]
}
},
"email": {
"type": "email"
},
"phone": {
"type": "phone"
}
}
},
"buildout_estimate": {
"type": "number"
},
"next_meeting_date": {
"type": "date"
}
}
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 · 6.86 s
{
"meeting_date": "2026-10-14",
"attendees": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+1 919 555 0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+1 919 555 0184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"buildout_estimate": 48500,
"next_meeting_date": "2026-10-28"
}Scoring detail
{
"fields": {
"meeting_date": {
"score": 1,
"expected": "2026-10-14",
"got": "2026-10-14"
},
"attendees": {
"score": 1,
"expected": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"got": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+1 919 555 0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+1 919 555 0184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
]
},
"buildout_estimate": {
"score": 1,
"expected": 48500,
"got": 48500
},
"next_meeting_date": {
"score": 1,
"expected": "2026-10-28",
"got": "2026-10-28"
}
},
"fields_correct": 4,
"fields_total": 4
}Ornith-1.5 9B
✗ fail · score 75 · 5.54 s
```json
{
"meeting_date": "2026-10-14",
"attendees": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+1 919 555 0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+1 919 555 0184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"buildout_estimate": 48500,
"next_meeting_date": null
}
```Scoring detail
{
"fields": {
"meeting_date": {
"score": 1,
"expected": "2026-10-14",
"got": "2026-10-14"
},
"attendees": {
"score": 1,
"expected": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"got": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+1 919 555 0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+1 919 555 0184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
]
},
"buildout_estimate": {
"score": 1,
"expected": 48500,
"got": 48500
},
"next_meeting_date": {
"score": 0,
"expected": "2026-10-28",
"got": null
}
},
"fields_correct": 3,
"fields_total": 4
}Granite 4.2 8B
✗ fail · score 88 · 5.74 s
{
"meeting_date": "2026-10-14",
"attendees": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19155550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19155550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"buildout_estimate": 48500,
"next_meeting_date": "2026-10-28"
}Scoring detail
{
"fields": {
"meeting_date": {
"score": 1,
"expected": "2026-10-14",
"got": "2026-10-14"
},
"attendees": {
"score": 0.5,
"expected": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"got": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19155550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19155550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
]
},
"buildout_estimate": {
"score": 1,
"expected": 48500,
"got": 48500
},
"next_meeting_date": {
"score": 1,
"expected": "2026-10-28",
"got": "2026-10-28"
}
},
"fields_correct": 3,
"fields_total": 4
}Spark-X2.5 4B
✓ pass · score 100 · 4.32 s
{
"meeting_date": "2026-10-14",
"attendees": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+1 919 555 0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+1 919 555 0184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"buildout_estimate": 48500,
"next_meeting_date": "2026-10-28"
}Scoring detail
{
"fields": {
"meeting_date": {
"score": 1,
"expected": "2026-10-14",
"got": "2026-10-14"
},
"attendees": {
"score": 1,
"expected": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"got": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+1 919 555 0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+1 919 555 0184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
]
},
"buildout_estimate": {
"score": 1,
"expected": 48500,
"got": 48500
},
"next_meeting_date": {
"score": 1,
"expected": "2026-10-28",
"got": "2026-10-28"
}
},
"fields_correct": 4,
"fields_total": 4
}Mellum2.1 12B-A2.5B
✓ pass · score 100 · 2.89 s
{
"meeting_date": "2026-10-14",
"attendees": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"buildout_estimate": 48500,
"next_meeting_date": "2026-10-28"
}Scoring detail
{
"fields": {
"meeting_date": {
"score": 1,
"expected": "2026-10-14",
"got": "2026-10-14"
},
"attendees": {
"score": 1,
"expected": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"got": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
]
},
"buildout_estimate": {
"score": 1,
"expected": 48500,
"got": 48500
},
"next_meeting_date": {
"score": 1,
"expected": "2026-10-28",
"got": "2026-10-28"
}
},
"fields_correct": 4,
"fields_total": 4
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 88 · 1.71 s
{"meeting_date": "2026-10-14", "attendees": [{"name": "Rosa Delgado", "email": "rosa.delgado@millbrookcoop.org", "phone": null}, {"name": "Tom Hirsch", "email": null, "phone": "919-555-0127"}, {"name": "Anjali Mehta", "email": "anjali.mehta@millbrookcoop.org", "phone": "919.555.0184"}, {"name": "Keith from facilities", "email": null, "phone": null}], "buildout_estimate": 48500, "next_meeting_date": "2026-10-28"}Scoring detail
{
"fields": {
"meeting_date": {
"score": 1,
"expected": "2026-10-14",
"got": "2026-10-14"
},
"attendees": {
"score": 0.5,
"expected": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"got": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "919-555-0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "919.555.0184"
},
{
"name": "Keith from facilities",
"email": null,
"phone": null
}
]
},
"buildout_estimate": {
"score": 1,
"expected": 48500,
"got": 48500
},
"next_meeting_date": {
"score": 1,
"expected": "2026-10-28",
"got": "2026-10-28"
}
},
"fields_correct": 3,
"fields_total": 4
}MiniCPM5 2B
✗ fail · score 63 · 1.96 s
{
"meeting_date": "2026-10-14",
"attendees": [
{"name": "Rosa Delgado", "email": "rosa.delgado@millbrookcoop.org", "phone": null},
{"name": "Tom Hirsch", "email": null, "phone": "919-555-0127"},
{"name": "Anjali Mehta", "email": "anjali.mehta@millbrookcoop.org", "phone": "919.555.0184"},
{"name": "Keith", "email": null, "phone": null}
],
"buildout_estimate": 48500,
"next_meeting_date": "2026-11-12"
}Scoring detail
{
"fields": {
"meeting_date": {
"score": 1,
"expected": "2026-10-14",
"got": "2026-10-14"
},
"attendees": {
"score": 0.5,
"expected": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"got": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "919-555-0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "919.555.0184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
]
},
"buildout_estimate": {
"score": 1,
"expected": 48500,
"got": 48500
},
"next_meeting_date": {
"score": 0,
"expected": "2026-10-28",
"got": "2026-11-12"
}
},
"fields_correct": 2,
"fields_total": 4
}K2-Horizon 0.9B
✗ fail · score 88 · 2.56 s
{
"meeting_date": "2026-10-14",
"attendees": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "919-555-0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "919.555.0184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"buildout_estimate": 48500,
"next_meeting_date": "2026-10-28"
}Scoring detail
{
"fields": {
"meeting_date": {
"score": 1,
"expected": "2026-10-14",
"got": "2026-10-14"
},
"attendees": {
"score": 0.5,
"expected": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "+19195550127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "+19195550184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
],
"got": [
{
"name": "Rosa Delgado",
"email": "rosa.delgado@millbrookcoop.org",
"phone": null
},
{
"name": "Tom Hirsch",
"email": null,
"phone": "919-555-0127"
},
{
"name": "Anjali Mehta",
"email": "anjali.mehta@millbrookcoop.org",
"phone": "919.555.0184"
},
{
"name": "Keith",
"email": null,
"phone": null
}
]
},
"buildout_estimate": {
"score": 1,
"expected": 48500,
"got": 48500
},
"next_meeting_date": {
"score": 1,
"expected": "2026-10-28",
"got": "2026-10-28"
}
},
"fields_correct": 3,
"fields_total": 4
}← summarize-extract-025All Summarize + extract itemssummarize-extract-027 →