Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract

summarize-extract-005

easy extractionreceipt

The prompt

user

Extract information from the fuel receipt below. Reply with only a JSON object with exactly these keys:
- "station" (string): station name without the store number
- "date" (string "YYYY-MM-DD"): purchase date
- "fuel_grade" (string): one of "regular", "midgrade", "premium", "diesel"
- "gallons" (number): gallons pumped
- "price_per_gallon" (number): price per gallon
- "total" (number): total charged

Rules: include every key listed. Use null for any field that is not stated in the text (do not guess). Dates are YYYY-MM-DD. Times are HH:MM in 24-hour format. Amounts and counts are plain JSON numbers (no currency symbols, no commas). Phone numbers are in international format with + and the country code, for example +1 312 555 0100; leave out any national trunk prefix 0, so (0)20 or 020 becomes 20 after +44.

Fuel receipt:
"""
QUIKFILL #2207
Rt 9 & Pine Hill Rd
Bellmore Township NJ
07/19/2026  06:55
PUMP 06  REGULAR
GALLONS      11.482
PRICE/GAL   $3.459
FUEL TOTAL  $39.72
CREDIT  AMEX  ....1008
THANK YOU - DRIVE SAFE
"""

What we expected

{
  "kind": "extract",
  "answer": {
    "station": "QuikFill",
    "date": "2026-07-19",
    "fuel_grade": "regular",
    "gallons": 11.482,
    "price_per_gallon": 3.459,
    "total": 39.72
  },
  "fields": {
    "station": {
      "type": "string",
      "accept": [
        "QuikFill #2207",
        "QuikFill 2207"
      ]
    },
    "date": {
      "type": "date"
    },
    "fuel_grade": {
      "type": "enum"
    },
    "gallons": {
      "type": "number"
    },
    "price_per_gallon": {
      "type": "number"
    },
    "total": {
      "type": "number"
    }
  }
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 2.55 s

{"station": "QUIKFILL", "date": "2026-07-19", "fuel_grade": "regular", "gallons": 11.482, "price_per_gallon": 3.459, "total": 39.72}

Scoring detail

{
  "fields": {
    "station": {
      "score": 1,
      "expected": "QuikFill",
      "got": "QUIKFILL"
    },
    "date": {
      "score": 1,
      "expected": "2026-07-19",
      "got": "2026-07-19"
    },
    "fuel_grade": {
      "score": 1,
      "expected": "regular",
      "got": "regular"
    },
    "gallons": {
      "score": 1,
      "expected": 11.482,
      "got": 11.482
    },
    "price_per_gallon": {
      "score": 1,
      "expected": 3.459,
      "got": 3.459
    },
    "total": {
      "score": 1,
      "expected": 39.72,
      "got": 39.72
    }
  },
  "fields_correct": 6,
  "fields_total": 6
}

Ornith-1.5 9B

✓ pass · score 100 · 2.19 s

{"station": "QUIKFILL #2207", "date": "2026-07-19", "fuel_grade": "regular", "gallons": 11.482, "price_per_gallon": 3.459, "total": 39.72}

Scoring detail

{
  "fields": {
    "station": {
      "score": 1,
      "expected": "QuikFill",
      "got": "QUIKFILL #2207"
    },
    "date": {
      "score": 1,
      "expected": "2026-07-19",
      "got": "2026-07-19"
    },
    "fuel_grade": {
      "score": 1,
      "expected": "regular",
      "got": "regular"
    },
    "gallons": {
      "score": 1,
      "expected": 11.482,
      "got": 11.482
    },
    "price_per_gallon": {
      "score": 1,
      "expected": 3.459,
      "got": 3.459
    },
    "total": {
      "score": 1,
      "expected": 39.72,
      "got": 39.72
    }
  },
  "fields_correct": 6,
  "fields_total": 6
}

Granite 4.2 8B

✓ pass · score 100 · 2.41 s

{
  "station": "QUIKFILL",
  "date": "2026-07-19",
  "fuel_grade": "regular",
  "gallons": 11.482,
  "price_per_gallon": 3.459,
  "total": 39.72
}

Scoring detail

{
  "fields": {
    "station": {
      "score": 1,
      "expected": "QuikFill",
      "got": "QUIKFILL"
    },
    "date": {
      "score": 1,
      "expected": "2026-07-19",
      "got": "2026-07-19"
    },
    "fuel_grade": {
      "score": 1,
      "expected": "regular",
      "got": "regular"
    },
    "gallons": {
      "score": 1,
      "expected": 11.482,
      "got": 11.482
    },
    "price_per_gallon": {
      "score": 1,
      "expected": 3.459,
      "got": 3.459
    },
    "total": {
      "score": 1,
      "expected": 39.72,
      "got": 39.72
    }
  },
  "fields_correct": 6,
  "fields_total": 6
}

Spark-X2.5 4B

✓ pass · score 100 · 2.02 s

{
  "station": "QUIKFILL",
  "date": "2026-07-19",
  "fuel_grade": "regular",
  "gallons": 11.482,
  "price_per_gallon": 3.459,
  "total": 39.72
}

Scoring detail

{
  "fields": {
    "station": {
      "score": 1,
      "expected": "QuikFill",
      "got": "QUIKFILL"
    },
    "date": {
      "score": 1,
      "expected": "2026-07-19",
      "got": "2026-07-19"
    },
    "fuel_grade": {
      "score": 1,
      "expected": "regular",
      "got": "regular"
    },
    "gallons": {
      "score": 1,
      "expected": 11.482,
      "got": 11.482
    },
    "price_per_gallon": {
      "score": 1,
      "expected": 3.459,
      "got": 3.459
    },
    "total": {
      "score": 1,
      "expected": 39.72,
      "got": 39.72
    }
  },
  "fields_correct": 6,
  "fields_total": 6
}

Mellum2.1 12B-A2.5B

✓ pass · score 100 · 1.09 s

{
  "station": "QUIKFILL",
  "date": "2026-07-19",
  "fuel_grade": "regular",
  "gallons": 11.482,
  "price_per_gallon": 3.459,
  "total": 39.72
}

Scoring detail

{
  "fields": {
    "station": {
      "score": 1,
      "expected": "QuikFill",
      "got": "QUIKFILL"
    },
    "date": {
      "score": 1,
      "expected": "2026-07-19",
      "got": "2026-07-19"
    },
    "fuel_grade": {
      "score": 1,
      "expected": "regular",
      "got": "regular"
    },
    "gallons": {
      "score": 1,
      "expected": 11.482,
      "got": 11.482
    },
    "price_per_gallon": {
      "score": 1,
      "expected": 3.459,
      "got": 3.459
    },
    "total": {
      "score": 1,
      "expected": 39.72,
      "got": 39.72
    }
  },
  "fields_correct": 6,
  "fields_total": 6
}

Ling-3.0-tiny (7.9B-A1.3B)

✓ pass · score 100 · 0.77 s

{"station": "QUIKFILL", "date": "2026-07-19", "fuel_grade": "regular", "gallons": 11.482, "price_per_gallon": 3.459, "total": 39.72}

Scoring detail

{
  "fields": {
    "station": {
      "score": 1,
      "expected": "QuikFill",
      "got": "QUIKFILL"
    },
    "date": {
      "score": 1,
      "expected": "2026-07-19",
      "got": "2026-07-19"
    },
    "fuel_grade": {
      "score": 1,
      "expected": "regular",
      "got": "regular"
    },
    "gallons": {
      "score": 1,
      "expected": 11.482,
      "got": 11.482
    },
    "price_per_gallon": {
      "score": 1,
      "expected": 3.459,
      "got": 3.459
    },
    "total": {
      "score": 1,
      "expected": 39.72,
      "got": 39.72
    }
  },
  "fields_correct": 6,
  "fields_total": 6
}

MiniCPM5 2B

✓ pass · score 100 · 0.95 s

{"station":"QUIKFILL","date":"2026-07-19","fuel_grade":"regular","gallons":11.482,"price_per_gallon":3.459,"total":39.72}

Scoring detail

{
  "fields": {
    "station": {
      "score": 1,
      "expected": "QuikFill",
      "got": "QUIKFILL"
    },
    "date": {
      "score": 1,
      "expected": "2026-07-19",
      "got": "2026-07-19"
    },
    "fuel_grade": {
      "score": 1,
      "expected": "regular",
      "got": "regular"
    },
    "gallons": {
      "score": 1,
      "expected": 11.482,
      "got": 11.482
    },
    "price_per_gallon": {
      "score": 1,
      "expected": 3.459,
      "got": 3.459
    },
    "total": {
      "score": 1,
      "expected": 39.72,
      "got": 39.72
    }
  },
  "fields_correct": 6,
  "fields_total": 6
}

K2-Horizon 0.9B

✗ fail · score 83 · 1.14 s

{
  "station": "QUICKFILL",
  "date": "2026-07-19",
  "fuel_grade": "regular",
  "gallons": 11.482,
  "price_per_gallon": 3.459,
  "total": 39.72
}

Scoring detail

{
  "fields": {
    "station": {
      "score": 0,
      "expected": "QuikFill",
      "got": "QUICKFILL"
    },
    "date": {
      "score": 1,
      "expected": "2026-07-19",
      "got": "2026-07-19"
    },
    "fuel_grade": {
      "score": 1,
      "expected": "regular",
      "got": "regular"
    },
    "gallons": {
      "score": 1,
      "expected": 11.482,
      "got": 11.482
    },
    "price_per_gallon": {
      "score": 1,
      "expected": 3.459,
      "got": 3.459
    },
    "total": {
      "score": 1,
      "expected": 39.72,
      "got": 39.72
    }
  },
  "fields_correct": 5,
  "fields_total": 6
}

← summarize-extract-004All Summarize + extract itemssummarize-extract-006 →