Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract
summarize-extract-001
easy extractionreceipt
The prompt
user
Extract information from the receipt below. Reply with only a JSON object with exactly these keys: - "merchant" (string): the business name - "date" (string "YYYY-MM-DD"): purchase date - "time" (string "HH:MM" (24-hour)): purchase time - "total" (number): the total charged - "card_last4" (string): last four digits of the card used, as a string Rules: include every key listed. Use null for any field that is not stated in the text (do not guess). Dates are YYYY-MM-DD. Times are HH:MM in 24-hour format. Amounts and counts are plain JSON numbers (no currency symbols, no commas). Phone numbers are in international format with + and the country code, for example +1 312 555 0100; leave out any national trunk prefix 0, so (0)20 or 020 becomes 20 after +44. Receipt: """ THE COPPER KETTLE CAFE 418 Alder Street, Portland OR 97205 (503) 555-0139 Order #2291 Server: Maya 03/14/2026 08:42 AM 1 Lg Oat Latte 5.75 1 Blueberry Scone 3.50 1 Drip Coffee Refill 1.00 Subtotal 10.25 Tax 0.00 TOTAL 10.25 VISA ************4417 Approved Auth 08812C Thank you! Come back soon :) """
What we expected
{
"kind": "extract",
"answer": {
"merchant": "The Copper Kettle Cafe",
"date": "2026-03-14",
"time": "08:42",
"total": 10.25,
"card_last4": "4417"
},
"fields": {
"merchant": {
"type": "string",
"accept": [
"Copper Kettle Cafe"
]
},
"date": {
"type": "date"
},
"time": {
"type": "time"
},
"total": {
"type": "number"
},
"card_last4": {
"type": "last4"
}
}
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 · 2.56 s
{"merchant": "THE COPPER KETTLE CAFE", "date": "2026-03-14", "time": "08:42", "total": 10.25, "card_last4": "4417"}Scoring detail
{
"fields": {
"merchant": {
"score": 1,
"expected": "The Copper Kettle Cafe",
"got": "THE COPPER KETTLE CAFE"
},
"date": {
"score": 1,
"expected": "2026-03-14",
"got": "2026-03-14"
},
"time": {
"score": 1,
"expected": "08:42",
"got": "08:42"
},
"total": {
"score": 1,
"expected": 10.25,
"got": 10.25
},
"card_last4": {
"score": 1,
"expected": "4417",
"got": "4417"
}
},
"fields_correct": 5,
"fields_total": 5
}Ornith-1.5 9B
✓ pass · score 100 · 2.47 s
{
"merchant": "THE COPPER KETTLE CAFE",
"date": "2026-03-14",
"time": "08:42",
"total": 10.25,
"card_last4": "4417"
}Scoring detail
{
"fields": {
"merchant": {
"score": 1,
"expected": "The Copper Kettle Cafe",
"got": "THE COPPER KETTLE CAFE"
},
"date": {
"score": 1,
"expected": "2026-03-14",
"got": "2026-03-14"
},
"time": {
"score": 1,
"expected": "08:42",
"got": "08:42"
},
"total": {
"score": 1,
"expected": 10.25,
"got": 10.25
},
"card_last4": {
"score": 1,
"expected": "4417",
"got": "4417"
}
},
"fields_correct": 5,
"fields_total": 5
}Granite 4.2 8B
✓ pass · score 100 · 2.35 s
{
"merchant": "THE COPPER KETTLE CAFE",
"date": "2026-03-14",
"time": "08:42",
"total": 10.25,
"card_last4": "4417"
}Scoring detail
{
"fields": {
"merchant": {
"score": 1,
"expected": "The Copper Kettle Cafe",
"got": "THE COPPER KETTLE CAFE"
},
"date": {
"score": 1,
"expected": "2026-03-14",
"got": "2026-03-14"
},
"time": {
"score": 1,
"expected": "08:42",
"got": "08:42"
},
"total": {
"score": 1,
"expected": 10.25,
"got": 10.25
},
"card_last4": {
"score": 1,
"expected": "4417",
"got": "4417"
}
},
"fields_correct": 5,
"fields_total": 5
}Spark-X2.5 4B
✓ pass · score 100 · 1.69 s
{
"merchant": "THE COPPER KETTLE CAFE",
"date": "2026-03-14",
"time": "08:42",
"total": 10.25,
"card_last4": "4417"
}Scoring detail
{
"fields": {
"merchant": {
"score": 1,
"expected": "The Copper Kettle Cafe",
"got": "THE COPPER KETTLE CAFE"
},
"date": {
"score": 1,
"expected": "2026-03-14",
"got": "2026-03-14"
},
"time": {
"score": 1,
"expected": "08:42",
"got": "08:42"
},
"total": {
"score": 1,
"expected": 10.25,
"got": 10.25
},
"card_last4": {
"score": 1,
"expected": "4417",
"got": "4417"
}
},
"fields_correct": 5,
"fields_total": 5
}Mellum2.1 12B-A2.5B
✓ pass · score 100 · 1.03 s
{
"merchant": "THE COPPER KETTLE CAFE",
"date": "2026-03-14",
"time": "08:42",
"total": 10.25,
"card_last4": "4417"
}Scoring detail
{
"fields": {
"merchant": {
"score": 1,
"expected": "The Copper Kettle Cafe",
"got": "THE COPPER KETTLE CAFE"
},
"date": {
"score": 1,
"expected": "2026-03-14",
"got": "2026-03-14"
},
"time": {
"score": 1,
"expected": "08:42",
"got": "08:42"
},
"total": {
"score": 1,
"expected": 10.25,
"got": 10.25
},
"card_last4": {
"score": 1,
"expected": "4417",
"got": "4417"
}
},
"fields_correct": 5,
"fields_total": 5
}Ling-3.0-tiny (7.9B-A1.3B)
✓ pass · score 100 · 0.69 s
{"merchant": "THE COPPER KETTLE CAFE", "date": "2026-03-14", "time": "08:42", "total": 10.25, "card_last4": "4417"}Scoring detail
{
"fields": {
"merchant": {
"score": 1,
"expected": "The Copper Kettle Cafe",
"got": "THE COPPER KETTLE CAFE"
},
"date": {
"score": 1,
"expected": "2026-03-14",
"got": "2026-03-14"
},
"time": {
"score": 1,
"expected": "08:42",
"got": "08:42"
},
"total": {
"score": 1,
"expected": 10.25,
"got": 10.25
},
"card_last4": {
"score": 1,
"expected": "4417",
"got": "4417"
}
},
"fields_correct": 5,
"fields_total": 5
}MiniCPM5 2B
✓ pass · score 100 · 0.87 s
{"merchant": "THE COPPER KETTLE CAFE", "date": "2026-03-14", "time": "08:42", "total": 10.25, "card_last4": "4417"}Scoring detail
{
"fields": {
"merchant": {
"score": 1,
"expected": "The Copper Kettle Cafe",
"got": "THE COPPER KETTLE CAFE"
},
"date": {
"score": 1,
"expected": "2026-03-14",
"got": "2026-03-14"
},
"time": {
"score": 1,
"expected": "08:42",
"got": "08:42"
},
"total": {
"score": 1,
"expected": 10.25,
"got": 10.25
},
"card_last4": {
"score": 1,
"expected": "4417",
"got": "4417"
}
},
"fields_correct": 5,
"fields_total": 5
}K2-Horizon 0.9B
✓ pass · score 100 · 1.14 s
{
"merchant": "THE COPPER KETTLE CAFE",
"date": "2026-03-14",
"time": "08:42",
"total": 10.25,
"card_last4": "4417"
}Scoring detail
{
"fields": {
"merchant": {
"score": 1,
"expected": "The Copper Kettle Cafe",
"got": "THE COPPER KETTLE CAFE"
},
"date": {
"score": 1,
"expected": "2026-03-14",
"got": "2026-03-14"
},
"time": {
"score": 1,
"expected": "08:42",
"got": "08:42"
},
"total": {
"score": 1,
"expected": 10.25,
"got": 10.25
},
"card_last4": {
"score": 1,
"expected": "4417",
"got": "4417"
}
},
"fields_correct": 5,
"fields_total": 5
}