Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract

summarize-extract-022

easy extractionemail

The prompt

user

Extract information from the appointment reminder below. Reply with only a JSON object with exactly these keys:
- "patient_first_name" (string): patient's first name
- "provider" (string): dentist's name
- "practice" (string): practice name
- "date" (string "YYYY-MM-DD"): appointment date
- "time" (string "HH:MM" (24-hour)): appointment time
- "arrive_early_minutes" (number): minutes early to arrive
- "cancellation_notice_hours" (number): hours of notice required to reschedule
- "late_cancel_fee" (number): late cancellation fee
- "phone" (string): office phone

Rules: include every key listed. Use null for any field that is not stated in the text (do not guess). Dates are YYYY-MM-DD. Times are HH:MM in 24-hour format. Amounts and counts are plain JSON numbers (no currency symbols, no commas). Phone numbers are in international format with + and the country code, for example +1 312 555 0100; leave out any national trunk prefix 0, so (0)20 or 020 becomes 20 after +44.

Appointment reminder:
"""
Hi Ana, this is a reminder of your appointment with Dr. Kwame Asante at Lakeside Family Dental on Wednesday, March 25, 2026 at 3:40 PM.

Please arrive 15 minutes early to update your paperwork.
Address: 2207 Shoreline Dr, Suite 110, Madison, WI 53704

Need to reschedule? Please give at least 48 hours' notice, or a $50 late-cancellation fee applies.
Reply C to confirm or call (608) 555-0131.
"""

What we expected

{
  "kind": "extract",
  "answer": {
    "patient_first_name": "Ana",
    "provider": "Dr. Kwame Asante",
    "practice": "Lakeside Family Dental",
    "date": "2026-03-25",
    "time": "15:40",
    "arrive_early_minutes": 15,
    "cancellation_notice_hours": 48,
    "late_cancel_fee": 50,
    "phone": "+16085550131"
  },
  "fields": {
    "patient_first_name": {
      "type": "string"
    },
    "provider": {
      "type": "string",
      "accept": [
        "Kwame Asante",
        "Dr. Asante",
        "Dr. Kwame Asante, DDS"
      ]
    },
    "practice": {
      "type": "string"
    },
    "date": {
      "type": "date"
    },
    "time": {
      "type": "time"
    },
    "arrive_early_minutes": {
      "type": "number"
    },
    "cancellation_notice_hours": {
      "type": "number"
    },
    "late_cancel_fee": {
      "type": "number"
    },
    "phone": {
      "type": "phone"
    }
  }
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 3.50 s

{"patient_first_name": "Ana", "provider": "Dr. Kwame Asante", "practice": "Lakeside Family Dental", "date": "2026-03-25", "time": "15:40", "arrive_early_minutes": 15, "cancellation_notice_hours": 48, "late_cancel_fee": 50, "phone": "+1 608 555 0131"}

Scoring detail

{
  "fields": {
    "patient_first_name": {
      "score": 1,
      "expected": "Ana",
      "got": "Ana"
    },
    "provider": {
      "score": 1,
      "expected": "Dr. Kwame Asante",
      "got": "Dr. Kwame Asante"
    },
    "practice": {
      "score": 1,
      "expected": "Lakeside Family Dental",
      "got": "Lakeside Family Dental"
    },
    "date": {
      "score": 1,
      "expected": "2026-03-25",
      "got": "2026-03-25"
    },
    "time": {
      "score": 1,
      "expected": "15:40",
      "got": "15:40"
    },
    "arrive_early_minutes": {
      "score": 1,
      "expected": 15,
      "got": 15
    },
    "cancellation_notice_hours": {
      "score": 1,
      "expected": 48,
      "got": 48
    },
    "late_cancel_fee": {
      "score": 1,
      "expected": 50,
      "got": 50
    },
    "phone": {
      "score": 1,
      "expected": "+16085550131",
      "got": "+1 608 555 0131"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Ornith-1.5 9B

✓ pass · score 100 · 3.83 s

```json
{
  "patient_first_name": "Ana",
  "provider": "Dr. Kwame Asante",
  "practice": "Lakeside Family Dental",
  "date": "2026-03-25",
  "time": "15:40",
  "arrive_early_minutes": 15,
  "cancellation_notice_hours": 48,
  "late_cancel_fee": 50,
  "phone": "+1 608 555 0131"
}
```

Scoring detail

{
  "fields": {
    "patient_first_name": {
      "score": 1,
      "expected": "Ana",
      "got": "Ana"
    },
    "provider": {
      "score": 1,
      "expected": "Dr. Kwame Asante",
      "got": "Dr. Kwame Asante"
    },
    "practice": {
      "score": 1,
      "expected": "Lakeside Family Dental",
      "got": "Lakeside Family Dental"
    },
    "date": {
      "score": 1,
      "expected": "2026-03-25",
      "got": "2026-03-25"
    },
    "time": {
      "score": 1,
      "expected": "15:40",
      "got": "15:40"
    },
    "arrive_early_minutes": {
      "score": 1,
      "expected": 15,
      "got": 15
    },
    "cancellation_notice_hours": {
      "score": 1,
      "expected": 48,
      "got": 48
    },
    "late_cancel_fee": {
      "score": 1,
      "expected": 50,
      "got": 50
    },
    "phone": {
      "score": 1,
      "expected": "+16085550131",
      "got": "+1 608 555 0131"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Granite 4.2 8B

✓ pass · score 100 · 3.56 s

{
  "patient_first_name": "Ana",
  "provider": "Dr. Kwame Asante",
  "practice": "Lakeside Family Dental",
  "date": "2026-03-25",
  "time": "15:40",
  "arrive_early_minutes": 15,
  "cancellation_notice_hours": 48,
  "late_cancel_fee": 50,
  "phone": "+1 608 555 0131"
}

Scoring detail

{
  "fields": {
    "patient_first_name": {
      "score": 1,
      "expected": "Ana",
      "got": "Ana"
    },
    "provider": {
      "score": 1,
      "expected": "Dr. Kwame Asante",
      "got": "Dr. Kwame Asante"
    },
    "practice": {
      "score": 1,
      "expected": "Lakeside Family Dental",
      "got": "Lakeside Family Dental"
    },
    "date": {
      "score": 1,
      "expected": "2026-03-25",
      "got": "2026-03-25"
    },
    "time": {
      "score": 1,
      "expected": "15:40",
      "got": "15:40"
    },
    "arrive_early_minutes": {
      "score": 1,
      "expected": 15,
      "got": 15
    },
    "cancellation_notice_hours": {
      "score": 1,
      "expected": 48,
      "got": 48
    },
    "late_cancel_fee": {
      "score": 1,
      "expected": 50,
      "got": 50
    },
    "phone": {
      "score": 1,
      "expected": "+16085550131",
      "got": "+1 608 555 0131"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Spark-X2.5 4B

✓ pass · score 100 · 2.59 s

{
  "patient_first_name": "Ana",
  "provider": "Dr. Kwame Asante",
  "practice": "Lakeside Family Dental",
  "date": "2026-03-25",
  "time": "15:40",
  "arrive_early_minutes": 15,
  "cancellation_notice_hours": 48,
  "late_cancel_fee": 50,
  "phone": "+1 608 555 0131"
}

Scoring detail

{
  "fields": {
    "patient_first_name": {
      "score": 1,
      "expected": "Ana",
      "got": "Ana"
    },
    "provider": {
      "score": 1,
      "expected": "Dr. Kwame Asante",
      "got": "Dr. Kwame Asante"
    },
    "practice": {
      "score": 1,
      "expected": "Lakeside Family Dental",
      "got": "Lakeside Family Dental"
    },
    "date": {
      "score": 1,
      "expected": "2026-03-25",
      "got": "2026-03-25"
    },
    "time": {
      "score": 1,
      "expected": "15:40",
      "got": "15:40"
    },
    "arrive_early_minutes": {
      "score": 1,
      "expected": 15,
      "got": 15
    },
    "cancellation_notice_hours": {
      "score": 1,
      "expected": 48,
      "got": 48
    },
    "late_cancel_fee": {
      "score": 1,
      "expected": 50,
      "got": 50
    },
    "phone": {
      "score": 1,
      "expected": "+16085550131",
      "got": "+1 608 555 0131"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Mellum2.1 12B-A2.5B

✓ pass · score 100 · 1.73 s

{
  "patient_first_name": "Ana",
  "provider": "Dr. Kwame Asante",
  "practice": "Lakeside Family Dental",
  "date": "2026-03-25",
  "time": "15:40",
  "arrive_early_minutes": 15,
  "cancellation_notice_hours": 48,
  "late_cancel_fee": 50,
  "phone": "+1 608 555 0131"
}

Scoring detail

{
  "fields": {
    "patient_first_name": {
      "score": 1,
      "expected": "Ana",
      "got": "Ana"
    },
    "provider": {
      "score": 1,
      "expected": "Dr. Kwame Asante",
      "got": "Dr. Kwame Asante"
    },
    "practice": {
      "score": 1,
      "expected": "Lakeside Family Dental",
      "got": "Lakeside Family Dental"
    },
    "date": {
      "score": 1,
      "expected": "2026-03-25",
      "got": "2026-03-25"
    },
    "time": {
      "score": 1,
      "expected": "15:40",
      "got": "15:40"
    },
    "arrive_early_minutes": {
      "score": 1,
      "expected": 15,
      "got": 15
    },
    "cancellation_notice_hours": {
      "score": 1,
      "expected": 48,
      "got": 48
    },
    "late_cancel_fee": {
      "score": 1,
      "expected": 50,
      "got": 50
    },
    "phone": {
      "score": 1,
      "expected": "+16085550131",
      "got": "+1 608 555 0131"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

Ling-3.0-tiny (7.9B-A1.3B)

✓ pass · score 100 · 1.15 s

{"patient_first_name": "Ana", "provider": "Dr. Kwame Asante", "practice": "Lakeside Family Dental", "date": "2026-03-25", "time": "15:40", "arrive_early_minutes": 15, "cancellation_notice_hours": 48, "late_cancel_fee": 50, "phone": "+1 608 555 0131"}

Scoring detail

{
  "fields": {
    "patient_first_name": {
      "score": 1,
      "expected": "Ana",
      "got": "Ana"
    },
    "provider": {
      "score": 1,
      "expected": "Dr. Kwame Asante",
      "got": "Dr. Kwame Asante"
    },
    "practice": {
      "score": 1,
      "expected": "Lakeside Family Dental",
      "got": "Lakeside Family Dental"
    },
    "date": {
      "score": 1,
      "expected": "2026-03-25",
      "got": "2026-03-25"
    },
    "time": {
      "score": 1,
      "expected": "15:40",
      "got": "15:40"
    },
    "arrive_early_minutes": {
      "score": 1,
      "expected": 15,
      "got": 15
    },
    "cancellation_notice_hours": {
      "score": 1,
      "expected": 48,
      "got": 48
    },
    "late_cancel_fee": {
      "score": 1,
      "expected": 50,
      "got": 50
    },
    "phone": {
      "score": 1,
      "expected": "+16085550131",
      "got": "+1 608 555 0131"
    }
  },
  "fields_correct": 9,
  "fields_total": 9
}

MiniCPM5 2B

✗ fail · score 89 · 1.35 s

{"patient_first_name": null, "provider": "Dr. Kwame Asante", "practice": "Lakeside Family Dental", "date": "2026-03-25", "time": "15:40", "arrive_early_minutes": 15, "cancellation_notice_hours": 48, "late_cancel_fee": 50, "phone": "+1 608 555 0131"}

Scoring detail

{
  "fields": {
    "patient_first_name": {
      "score": 0,
      "expected": "Ana",
      "got": null
    },
    "provider": {
      "score": 1,
      "expected": "Dr. Kwame Asante",
      "got": "Dr. Kwame Asante"
    },
    "practice": {
      "score": 1,
      "expected": "Lakeside Family Dental",
      "got": "Lakeside Family Dental"
    },
    "date": {
      "score": 1,
      "expected": "2026-03-25",
      "got": "2026-03-25"
    },
    "time": {
      "score": 1,
      "expected": "15:40",
      "got": "15:40"
    },
    "arrive_early_minutes": {
      "score": 1,
      "expected": 15,
      "got": 15
    },
    "cancellation_notice_hours": {
      "score": 1,
      "expected": 48,
      "got": 48
    },
    "late_cancel_fee": {
      "score": 1,
      "expected": 50,
      "got": 50
    },
    "phone": {
      "score": 1,
      "expected": "+16085550131",
      "got": "+1 608 555 0131"
    }
  },
  "fields_correct": 8,
  "fields_total": 9
}

K2-Horizon 0.9B

✗ fail · score 89 · 0.63 s

{
  "patient_first_name": "Ana",
  "provider": "Dr. Kwame Asante",
  "practice": "Lakeside Family Dental",
  "date": "2026-03-25",
  "time": "15:40",
  "arrive_early_minutes": 15,
  "cancellation_notice_hours": 48,
  "late_cancel_fee": 50,
  "phone": "+60805550131"
}

Scoring detail

{
  "fields": {
    "patient_first_name": {
      "score": 1,
      "expected": "Ana",
      "got": "Ana"
    },
    "provider": {
      "score": 1,
      "expected": "Dr. Kwame Asante",
      "got": "Dr. Kwame Asante"
    },
    "practice": {
      "score": 1,
      "expected": "Lakeside Family Dental",
      "got": "Lakeside Family Dental"
    },
    "date": {
      "score": 1,
      "expected": "2026-03-25",
      "got": "2026-03-25"
    },
    "time": {
      "score": 1,
      "expected": "15:40",
      "got": "15:40"
    },
    "arrive_early_minutes": {
      "score": 1,
      "expected": 15,
      "got": 15
    },
    "cancellation_notice_hours": {
      "score": 1,
      "expected": 48,
      "got": 48
    },
    "late_cancel_fee": {
      "score": 1,
      "expected": 50,
      "got": 50
    },
    "phone": {
      "score": 0,
      "expected": "+16085550131",
      "got": "+60805550131"
    }
  },
  "fields_correct": 8,
  "fields_total": 9
}

← summarize-extract-021All Summarize + extract itemssummarize-extract-023 →