Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract

summarize-extract-004

hard extractionreceipt

The prompt

user

Extract information from the restaurant receipt below. Reply with only a JSON object with exactly these keys:
- "merchant" (string): restaurant name
- "date" (string "YYYY-MM-DD"): date of the meal
- "time" (string "HH:MM" (24-hour)): time on the check
- "guests" (number): number of guests
- "subtotal" (number): food and drink subtotal before tax
- "tax" (number): tax amount
- "tip" (number): tip actually added by the customer
- "total" (number): final amount charged including tip
- "card_last4" (string): last four digits of the card, as a string

Rules: include every key listed. Use null for any field that is not stated in the text (do not guess). Dates are YYYY-MM-DD. Times are HH:MM in 24-hour format. Amounts and counts are plain JSON numbers (no currency symbols, no commas). Phone numbers are in international format with + and the country code, for example +1 312 555 0100; leave out any national trunk prefix 0, so (0)20 or 020 becomes 20 after +44.

Restaurant receipt:
"""
   OSTERIA LUNA
   1120 Grand Blvd, Kansas City MO 64106
   Tbl 14   Guests 3   Srv: Tomasz
   Chk 50873        Sep 26 '26  9:17 PM

   2 Margherita Pizza      @16.00   32.00
   1 Caesar Salad                    11.50
   1 Tiramisu                         8.00
   2 House Red (glass)     @ 9.00   18.00

   Subtotal                         69.50
   Tax                               6.12
   Amount Due                       75.62

   -- Suggested Gratuity --
   18%  =  12.51     20%  =  13.90     22%  =  15.29

   MASTERCARD XXXX XXXX XXXX 0093
   Amount:      75.62
   + Tip:       14.00
   = Total:     89.62

   Signature x________________
"""

What we expected

{
  "kind": "extract",
  "answer": {
    "merchant": "Osteria Luna",
    "date": "2026-09-26",
    "time": "21:17",
    "guests": 3,
    "subtotal": 69.5,
    "tax": 6.12,
    "tip": 14,
    "total": 89.62,
    "card_last4": "0093"
  },
  "fields": {
    "merchant": {
      "type": "string"
    },
    "date": {
      "type": "date"
    },
    "time": {
      "type": "time"
    },
    "guests": {
      "type": "number"
    },
    "subtotal": {
      "type": "number"
    },
    "tax": {
      "type": "number"
    },
    "tip": {
      "type": "number"
    },
    "total": {
      "type": "number"
    },
    "card_last4": {
      "type": "last4"
    }
  }
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 3.85 s

{"merchant": "OSTERIA LUNA", "date": "2026-09-26", "time": "21:17", "guests": 3, "subtotal": 69.5, "tax": 6.12, "tip": 14.0, "total": 89.62, "card_last4": "0093"}

Scoring detail

{
  "fields": {
    "merchant": {
      "score": 1,
      "expected": "Osteria Luna",
      "got": "OSTERIA LUNA"
    },
    "date": {
      "score": 1,
      "expected": "2026-09-26",
      "got": "2026-09-26"
    },
    "time": {
      "score": 1,
      "expected": "21:17",
      "got": "21:17"
    },
    "guests": {
      "score": 1,
      "expected": 3,
      "got": 3
    },
    "subtotal": {
      "score": 1,
      "expected": 69.5,
      "got": 69.5
    },
    "tax": {
      "score": 1,
      "expected": 6.12,
      "got": 6.12
    },
    "tip": {
      "score": 1,
      "expected": 14,
      "got": 14
    },
    "total": {
      "score": 1,
      "expected": 89.62,
      "got": 89.62
    },
    "card_last4": {
      "score": 1,
      "expected": "0093",
      "got": "0093"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Ornith-1.5 9B

✓ pass · score 100 · 3.44 s

{
  "merchant": "OSTERIA LUNA",
  "date": "2026-09-26",
  "time": "21:17",
  "guests": 3,
  "subtotal": 69.5,
  "tax": 6.12,
  "tip": 14.0,
  "total": 89.62,
  "card_last4": "0093"
}

Scoring detail

{
  "fields": {
    "merchant": {
      "score": 1,
      "expected": "Osteria Luna",
      "got": "OSTERIA LUNA"
    },
    "date": {
      "score": 1,
      "expected": "2026-09-26",
      "got": "2026-09-26"
    },
    "time": {
      "score": 1,
      "expected": "21:17",
      "got": "21:17"
    },
    "guests": {
      "score": 1,
      "expected": 3,
      "got": 3
    },
    "subtotal": {
      "score": 1,
      "expected": 69.5,
      "got": 69.5
    },
    "tax": {
      "score": 1,
      "expected": 6.12,
      "got": 6.12
    },
    "tip": {
      "score": 1,
      "expected": 14,
      "got": 14
    },
    "total": {
      "score": 1,
      "expected": 89.62,
      "got": 89.62
    },
    "card_last4": {
      "score": 1,
      "expected": "0093",
      "got": "0093"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Granite 4.2 8B

✓ pass · score 100 · 3.47 s

{
  "merchant": "OSTERIA LUNA",
  "date": "2026-09-26",
  "time": "21:17",
  "guests": 3,
  "subtotal": 69.5,
  "tax": 6.12,
  "tip": 14.0,
  "total": 89.62,
  "card_last4": "0093"
}

Scoring detail

{
  "fields": {
    "merchant": {
      "score": 1,
      "expected": "Osteria Luna",
      "got": "OSTERIA LUNA"
    },
    "date": {
      "score": 1,
      "expected": "2026-09-26",
      "got": "2026-09-26"
    },
    "time": {
      "score": 1,
      "expected": "21:17",
      "got": "21:17"
    },
    "guests": {
      "score": 1,
      "expected": 3,
      "got": 3
    },
    "subtotal": {
      "score": 1,
      "expected": 69.5,
      "got": 69.5
    },
    "tax": {
      "score": 1,
      "expected": 6.12,
      "got": 6.12
    },
    "tip": {
      "score": 1,
      "expected": 14,
      "got": 14
    },
    "total": {
      "score": 1,
      "expected": 89.62,
      "got": 89.62
    },
    "card_last4": {
      "score": 1,
      "expected": "0093",
      "got": "0093"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Spark-X2.5 4B

✓ pass · score 100 · 2.58 s

{
  "merchant": "OSTERIA LUNA",
  "date": "2026-09-26",
  "time": "21:17",
  "guests": 3,
  "subtotal": 69.5,
  "tax": 6.12,
  "tip": 14.0,
  "total": 89.62,
  "card_last4": "0093"
}

Scoring detail

{
  "fields": {
    "merchant": {
      "score": 1,
      "expected": "Osteria Luna",
      "got": "OSTERIA LUNA"
    },
    "date": {
      "score": 1,
      "expected": "2026-09-26",
      "got": "2026-09-26"
    },
    "time": {
      "score": 1,
      "expected": "21:17",
      "got": "21:17"
    },
    "guests": {
      "score": 1,
      "expected": 3,
      "got": 3
    },
    "subtotal": {
      "score": 1,
      "expected": 69.5,
      "got": 69.5
    },
    "tax": {
      "score": 1,
      "expected": 6.12,
      "got": 6.12
    },
    "tip": {
      "score": 1,
      "expected": 14,
      "got": 14
    },
    "total": {
      "score": 1,
      "expected": 89.62,
      "got": 89.62
    },
    "card_last4": {
      "score": 1,
      "expected": "0093",
      "got": "0093"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Mellum2.1 12B-A2.5B

✓ pass · score 100 · 1.60 s

{
  "merchant": "OSTERIA LUNA",
  "date": "2026-09-26",
  "time": "21:17",
  "guests": 3,
  "subtotal": 69.5,
  "tax": 6.12,
  "tip": 14.0,
  "total": 89.62,
  "card_last4": "0093"
}

Scoring detail

{
  "fields": {
    "merchant": {
      "score": 1,
      "expected": "Osteria Luna",
      "got": "OSTERIA LUNA"
    },
    "date": {
      "score": 1,
      "expected": "2026-09-26",
      "got": "2026-09-26"
    },
    "time": {
      "score": 1,
      "expected": "21:17",
      "got": "21:17"
    },
    "guests": {
      "score": 1,
      "expected": 3,
      "got": 3
    },
    "subtotal": {
      "score": 1,
      "expected": 69.5,
      "got": 69.5
    },
    "tax": {
      "score": 1,
      "expected": 6.12,
      "got": 6.12
    },
    "tip": {
      "score": 1,
      "expected": 14,
      "got": 14
    },
    "total": {
      "score": 1,
      "expected": 89.62,
      "got": 89.62
    },
    "card_last4": {
      "score": 1,
      "expected": "0093",
      "got": "0093"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Ling-3.0-tiny (7.9B-A1.3B)

✓ pass · score 100 · 1.08 s

{"merchant": "OSTERIA LUNA", "date": "2026-09-26", "time": "21:17", "guests": 3, "subtotal": 69.5, "tax": 6.12, "tip": 14.0, "total": 89.62, "card_last4": "0093"}

Scoring detail

{
  "fields": {
    "merchant": {
      "score": 1,
      "expected": "Osteria Luna",
      "got": "OSTERIA LUNA"
    },
    "date": {
      "score": 1,
      "expected": "2026-09-26",
      "got": "2026-09-26"
    },
    "time": {
      "score": 1,
      "expected": "21:17",
      "got": "21:17"
    },
    "guests": {
      "score": 1,
      "expected": 3,
      "got": 3
    },
    "subtotal": {
      "score": 1,
      "expected": 69.5,
      "got": 69.5
    },
    "tax": {
      "score": 1,
      "expected": 6.12,
      "got": 6.12
    },
    "tip": {
      "score": 1,
      "expected": 14,
      "got": 14
    },
    "total": {
      "score": 1,
      "expected": 89.62,
      "got": 89.62
    },
    "card_last4": {
      "score": 1,
      "expected": "0093",
      "got": "0093"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

MiniCPM5 2B

✓ pass · score 100 · 1.38 s

{"merchant": "OSTERIA LUNA", "date": "2026-09-26", "time": "21:17", "guests": 3, "subtotal": 69.5, "tax": 6.12, "tip": 14.0, "total": 89.62, "card_last4": "0093"}

Scoring detail

{
  "fields": {
    "merchant": {
      "score": 1,
      "expected": "Osteria Luna",
      "got": "OSTERIA LUNA"
    },
    "date": {
      "score": 1,
      "expected": "2026-09-26",
      "got": "2026-09-26"
    },
    "time": {
      "score": 1,
      "expected": "21:17",
      "got": "21:17"
    },
    "guests": {
      "score": 1,
      "expected": 3,
      "got": 3
    },
    "subtotal": {
      "score": 1,
      "expected": 69.5,
      "got": 69.5
    },
    "tax": {
      "score": 1,
      "expected": 6.12,
      "got": 6.12
    },
    "tip": {
      "score": 1,
      "expected": 14,
      "got": 14
    },
    "total": {
      "score": 1,
      "expected": 89.62,
      "got": 89.62
    },
    "card_last4": {
      "score": 1,
      "expected": "0093",
      "got": "0093"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

K2-Horizon 0.9B

✓ pass · score 100 · 1.48 s

{
  "merchant": "OSTERIA LUNA",
  "date": "2026-09-26",
  "time": "21:17",
  "guests": 3,
  "subtotal": 69.5,
  "tax": 6.12,
  "tip": 14.0,
  "total": 89.62,
  "card_last4": "0093"
}

Scoring detail

{
  "fields": {
    "merchant": {
      "score": 1,
      "expected": "Osteria Luna",
      "got": "OSTERIA LUNA"
    },
    "date": {
      "score": 1,
      "expected": "2026-09-26",
      "got": "2026-09-26"
    },
    "time": {
      "score": 1,
      "expected": "21:17",
      "got": "21:17"
    },
    "guests": {
      "score": 1,
      "expected": 3,
      "got": 3
    },
    "subtotal": {
      "score": 1,
      "expected": 69.5,
      "got": 69.5
    },
    "tax": {
      "score": 1,
      "expected": 6.12,
      "got": 6.12
    },
    "tip": {
      "score": 1,
      "expected": 14,
      "got": 14
    },
    "total": {
      "score": 1,
      "expected": 89.62,
      "got": 89.62
    },
    "card_last4": {
      "score": 1,
      "expected": "0093",
      "got": "0093"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

← summarize-extract-003All Summarize + extract itemssummarize-extract-005 →