Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract
summarize-extract-012
medium extractioninvite
The prompt
user
Extract information from the email below. Reply with only a JSON object with exactly these keys: - "title" (string): meeting name - "date" (string "YYYY-MM-DD"): meeting date - "start_time" (string "HH:MM" (24-hour)): start time - "end_time" (string "HH:MM" (24-hour)): end time (work it out from the stated length) - "timezone" (string): time zone abbreviation as written - "location" (string): room name - "organizer_email" (string): organizer's email address Rules: include every key listed. Use null for any field that is not stated in the text (do not guess). Dates are YYYY-MM-DD. Times are HH:MM in 24-hour format. Amounts and counts are plain JSON numbers (no currency symbols, no commas). Phone numbers are in international format with + and the country code, for example +1 312 555 0100; leave out any national trunk prefix 0, so (0)20 or 020 becomes 20 after +44. Email: """ From: Ingrid Solberg <isolberg@meridianfreight.com> To: Ops Leads Date: Mon, 19 Oct 2026 15:22:40 -0400 Subject: Quarterly vendor review - tomorrow Hi all, Quick heads-up: the quarterly vendor review got pulled forward, so we're doing it tomorrow at 10:30 ET. It should run about 90 minutes. I've booked Conference Room 4B on the third floor; remote folks can use the usual Teams link. Please bring your Q3 on-time delivery numbers for your lanes. Thanks, Ingrid """
What we expected
{
"kind": "extract",
"answer": {
"title": "Quarterly vendor review",
"date": "2026-10-20",
"start_time": "10:30",
"end_time": "12:00",
"timezone": "ET",
"location": "Conference Room 4B",
"organizer_email": "isolberg@meridianfreight.com"
},
"fields": {
"title": {
"type": "string",
"accept": [
"Quarterly vendor review meeting"
]
},
"date": {
"type": "date"
},
"start_time": {
"type": "time"
},
"end_time": {
"type": "time"
},
"timezone": {
"type": "string"
},
"location": {
"type": "string",
"accept": [
"Conference Room 4B, third floor",
"Conference Room 4B, 3rd floor",
"Conference Room 4B third floor",
"Room 4B",
"4B"
]
},
"organizer_email": {
"type": "email"
}
}
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 · 3.18 s
{"title": "Quarterly vendor review", "date": "2026-10-20", "start_time": "10:30", "end_time": "12:00", "timezone": "ET", "location": "Conference Room 4B", "organizer_email": "isolberg@meridianfreight.com"}Scoring detail
{
"fields": {
"title": {
"score": 1,
"expected": "Quarterly vendor review",
"got": "Quarterly vendor review"
},
"date": {
"score": 1,
"expected": "2026-10-20",
"got": "2026-10-20"
},
"start_time": {
"score": 1,
"expected": "10:30",
"got": "10:30"
},
"end_time": {
"score": 1,
"expected": "12:00",
"got": "12:00"
},
"timezone": {
"score": 1,
"expected": "ET",
"got": "ET"
},
"location": {
"score": 1,
"expected": "Conference Room 4B",
"got": "Conference Room 4B"
},
"organizer_email": {
"score": 1,
"expected": "isolberg@meridianfreight.com",
"got": "isolberg@meridianfreight.com"
}
},
"fields_correct": 7,
"fields_total": 7
}Ornith-1.5 9B
✓ pass · score 100 · 3.01 s
{
"title": "Quarterly vendor review",
"date": "2026-10-20",
"start_time": "10:30",
"end_time": "12:00",
"timezone": "ET",
"location": "Conference Room 4B",
"organizer_email": "isolberg@meridianfreight.com"
}Scoring detail
{
"fields": {
"title": {
"score": 1,
"expected": "Quarterly vendor review",
"got": "Quarterly vendor review"
},
"date": {
"score": 1,
"expected": "2026-10-20",
"got": "2026-10-20"
},
"start_time": {
"score": 1,
"expected": "10:30",
"got": "10:30"
},
"end_time": {
"score": 1,
"expected": "12:00",
"got": "12:00"
},
"timezone": {
"score": 1,
"expected": "ET",
"got": "ET"
},
"location": {
"score": 1,
"expected": "Conference Room 4B",
"got": "Conference Room 4B"
},
"organizer_email": {
"score": 1,
"expected": "isolberg@meridianfreight.com",
"got": "isolberg@meridianfreight.com"
}
},
"fields_correct": 7,
"fields_total": 7
}Granite 4.2 8B
✓ pass · score 100 · 2.93 s
{
"title": "Quarterly vendor review",
"date": "2026-10-20",
"start_time": "10:30",
"end_time": "12:00",
"timezone": "ET",
"location": "Conference Room 4B",
"organizer_email": "isolberg@meridianfreight.com"
}Scoring detail
{
"fields": {
"title": {
"score": 1,
"expected": "Quarterly vendor review",
"got": "Quarterly vendor review"
},
"date": {
"score": 1,
"expected": "2026-10-20",
"got": "2026-10-20"
},
"start_time": {
"score": 1,
"expected": "10:30",
"got": "10:30"
},
"end_time": {
"score": 1,
"expected": "12:00",
"got": "12:00"
},
"timezone": {
"score": 1,
"expected": "ET",
"got": "ET"
},
"location": {
"score": 1,
"expected": "Conference Room 4B",
"got": "Conference Room 4B"
},
"organizer_email": {
"score": 1,
"expected": "isolberg@meridianfreight.com",
"got": "isolberg@meridianfreight.com"
}
},
"fields_correct": 7,
"fields_total": 7
}Spark-X2.5 4B
✗ fail · score 86 · 2.07 s
{
"title": "Quarterly vendor review",
"date": "2026-10-19",
"start_time": "10:30",
"end_time": "12:00",
"timezone": "ET",
"location": "Conference Room 4B",
"organizer_email": "isolberg@meridianfreight.com"
}Scoring detail
{
"fields": {
"title": {
"score": 1,
"expected": "Quarterly vendor review",
"got": "Quarterly vendor review"
},
"date": {
"score": 0,
"expected": "2026-10-20",
"got": "2026-10-19"
},
"start_time": {
"score": 1,
"expected": "10:30",
"got": "10:30"
},
"end_time": {
"score": 1,
"expected": "12:00",
"got": "12:00"
},
"timezone": {
"score": 1,
"expected": "ET",
"got": "ET"
},
"location": {
"score": 1,
"expected": "Conference Room 4B",
"got": "Conference Room 4B"
},
"organizer_email": {
"score": 1,
"expected": "isolberg@meridianfreight.com",
"got": "isolberg@meridianfreight.com"
}
},
"fields_correct": 6,
"fields_total": 7
}Mellum2.1 12B-A2.5B
✓ pass · score 100 · 1.44 s
{
"title": "Quarterly vendor review",
"date": "2026-10-20",
"start_time": "10:30",
"end_time": "12:00",
"timezone": "ET",
"location": "Conference Room 4B",
"organizer_email": "isolberg@meridianfreight.com"
}Scoring detail
{
"fields": {
"title": {
"score": 1,
"expected": "Quarterly vendor review",
"got": "Quarterly vendor review"
},
"date": {
"score": 1,
"expected": "2026-10-20",
"got": "2026-10-20"
},
"start_time": {
"score": 1,
"expected": "10:30",
"got": "10:30"
},
"end_time": {
"score": 1,
"expected": "12:00",
"got": "12:00"
},
"timezone": {
"score": 1,
"expected": "ET",
"got": "ET"
},
"location": {
"score": 1,
"expected": "Conference Room 4B",
"got": "Conference Room 4B"
},
"organizer_email": {
"score": 1,
"expected": "isolberg@meridianfreight.com",
"got": "isolberg@meridianfreight.com"
}
},
"fields_correct": 7,
"fields_total": 7
}Ling-3.0-tiny (7.9B-A1.3B)
✓ pass · score 100 · 0.88 s
{"title": "Quarterly vendor review", "date": "2026-10-20", "start_time": "10:30", "end_time": "12:00", "timezone": "ET", "location": "Conference Room 4B", "organizer_email": "isolberg@meridianfreight.com"}Scoring detail
{
"fields": {
"title": {
"score": 1,
"expected": "Quarterly vendor review",
"got": "Quarterly vendor review"
},
"date": {
"score": 1,
"expected": "2026-10-20",
"got": "2026-10-20"
},
"start_time": {
"score": 1,
"expected": "10:30",
"got": "10:30"
},
"end_time": {
"score": 1,
"expected": "12:00",
"got": "12:00"
},
"timezone": {
"score": 1,
"expected": "ET",
"got": "ET"
},
"location": {
"score": 1,
"expected": "Conference Room 4B",
"got": "Conference Room 4B"
},
"organizer_email": {
"score": 1,
"expected": "isolberg@meridianfreight.com",
"got": "isolberg@meridianfreight.com"
}
},
"fields_correct": 7,
"fields_total": 7
}MiniCPM5 2B
✗ fail · score 86 · 1.12 s
{"title": "Quarterly vendor review", "date": "2026-10-20", "start_time": "10:30", "end_time": "11:00", "timezone": "ET", "location": "Conference Room 4B", "organizer_email": "isolberg@meridianfreight.com"}Scoring detail
{
"fields": {
"title": {
"score": 1,
"expected": "Quarterly vendor review",
"got": "Quarterly vendor review"
},
"date": {
"score": 1,
"expected": "2026-10-20",
"got": "2026-10-20"
},
"start_time": {
"score": 1,
"expected": "10:30",
"got": "10:30"
},
"end_time": {
"score": 0,
"expected": "12:00",
"got": "11:00"
},
"timezone": {
"score": 1,
"expected": "ET",
"got": "ET"
},
"location": {
"score": 1,
"expected": "Conference Room 4B",
"got": "Conference Room 4B"
},
"organizer_email": {
"score": 1,
"expected": "isolberg@meridianfreight.com",
"got": "isolberg@meridianfreight.com"
}
},
"fields_correct": 6,
"fields_total": 7
}K2-Horizon 0.9B
✗ fail · score 57 · 0.52 s
{
"title": "Quarterly vendor review",
"date": "2026-10-19",
"start_time": "10:30",
"end_time": "11:20",
"timezone": "-0400",
"location": "Conference Room 4B",
"organizer_email": "isolberg@meridianfreight.com"
}Scoring detail
{
"fields": {
"title": {
"score": 1,
"expected": "Quarterly vendor review",
"got": "Quarterly vendor review"
},
"date": {
"score": 0,
"expected": "2026-10-20",
"got": "2026-10-19"
},
"start_time": {
"score": 1,
"expected": "10:30",
"got": "10:30"
},
"end_time": {
"score": 0,
"expected": "12:00",
"got": "11:20"
},
"timezone": {
"score": 0,
"expected": "ET",
"got": "-0400"
},
"location": {
"score": 1,
"expected": "Conference Room 4B",
"got": "Conference Room 4B"
},
"organizer_email": {
"score": 1,
"expected": "isolberg@meridianfreight.com",
"got": "isolberg@meridianfreight.com"
}
},
"fields_correct": 4,
"fields_total": 7
}← summarize-extract-011All Summarize + extract itemssummarize-extract-013 →