Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract

summarize-extract-020

hard extractionsignature

The prompt

user

Extract information from the email below. Reply with only a JSON object with exactly these keys:
- "name" (string): name of the person who wrote the newest message
- "title" (string): their job title
- "company" (string): their company
- "office_phone" (string): their office/landline number
- "mobile_phone" (string): their mobile number
- "email" (string): their email
- "city" (string): their city
- "postal_code" (string): their postcode

Rules: include every key listed. Use null for any field that is not stated in the text (do not guess). Dates are YYYY-MM-DD. Times are HH:MM in 24-hour format. Amounts and counts are plain JSON numbers (no currency symbols, no commas). Phone numbers are in international format with + and the country code, for example +1 312 555 0100; leave out any national trunk prefix 0, so (0)20 or 020 becomes 20 after +44.

Email:
"""
Hi Ben,

That works for us - I'll send the revised floor plan by Friday, and we can walk the site on the 20th.

Kind regards,
Fiona Achterberg
Project Lead, Interiors
Haverstock & Pryce Ltd
T +44 (0)20 7946 0321 | M +44 (0)7700 900418
fiona.achterberg@haverstockpryce.co.uk
14 Calder Mews, London N1 7QT

> On 12 Oct 2026, at 15:02, Ben Castellano <ben@castellanohomes.com> wrote:
>
> Hi Fiona, can we move the walkthrough to the 20th? Mornings are best for me.
>
> Ben Castellano | Owner | Castellano Homes
> +1 (617) 555-0148 | castellanohomes.com
"""

What we expected

{
  "kind": "extract",
  "answer": {
    "name": "Fiona Achterberg",
    "title": "Project Lead, Interiors",
    "company": "Haverstock & Pryce Ltd",
    "office_phone": "+442079460321",
    "mobile_phone": "+447700900418",
    "email": "fiona.achterberg@haverstockpryce.co.uk",
    "city": "London",
    "postal_code": "N1 7QT"
  },
  "fields": {
    "name": {
      "type": "string"
    },
    "title": {
      "type": "string",
      "accept": [
        "Project Lead",
        "Project Lead - Interiors",
        "Interiors Project Lead"
      ]
    },
    "company": {
      "type": "string",
      "accept": [
        "Haverstock and Pryce Ltd",
        "Haverstock & Pryce",
        "Haverstock & Pryce Limited"
      ]
    },
    "office_phone": {
      "type": "phone"
    },
    "mobile_phone": {
      "type": "phone"
    },
    "email": {
      "type": "email"
    },
    "city": {
      "type": "string"
    },
    "postal_code": {
      "type": "code"
    }
  }
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 4.12 s

{"name": "Fiona Achterberg", "title": "Project Lead, Interiors", "company": "Haverstock & Pryce Ltd", "office_phone": "+44 20 7946 0321", "mobile_phone": "+44 7700 900418", "email": "fiona.achterberg@haverstockpryce.co.uk", "city": "London", "postal_code": "N1 7QT"}

Scoring detail

{
  "fields": {
    "name": {
      "score": 1,
      "expected": "Fiona Achterberg",
      "got": "Fiona Achterberg"
    },
    "title": {
      "score": 1,
      "expected": "Project Lead, Interiors",
      "got": "Project Lead, Interiors"
    },
    "company": {
      "score": 1,
      "expected": "Haverstock & Pryce Ltd",
      "got": "Haverstock & Pryce Ltd"
    },
    "office_phone": {
      "score": 1,
      "expected": "+442079460321",
      "got": "+44 20 7946 0321"
    },
    "mobile_phone": {
      "score": 1,
      "expected": "+447700900418",
      "got": "+44 7700 900418"
    },
    "email": {
      "score": 1,
      "expected": "fiona.achterberg@haverstockpryce.co.uk",
      "got": "fiona.achterberg@haverstockpryce.co.uk"
    },
    "city": {
      "score": 1,
      "expected": "London",
      "got": "London"
    },
    "postal_code": {
      "score": 1,
      "expected": "N1 7QT",
      "got": "N1 7QT"
    }
  },
  "fields_correct": 8,
  "fields_total": 8
}

Ornith-1.5 9B

✓ pass · score 100 · 4.03 s

```json
{
  "name": "Fiona Achterberg",
  "title": "Project Lead, Interiors",
  "company": "Haverstock & Pryce Ltd",
  "office_phone": "+44 20 7946 0321",
  "mobile_phone": "+44 7700 900418",
  "email": "fiona.achterberg@haverstockpryce.co.uk",
  "city": "London",
  "postal_code": "N1 7QT"
}
```

Scoring detail

{
  "fields": {
    "name": {
      "score": 1,
      "expected": "Fiona Achterberg",
      "got": "Fiona Achterberg"
    },
    "title": {
      "score": 1,
      "expected": "Project Lead, Interiors",
      "got": "Project Lead, Interiors"
    },
    "company": {
      "score": 1,
      "expected": "Haverstock & Pryce Ltd",
      "got": "Haverstock & Pryce Ltd"
    },
    "office_phone": {
      "score": 1,
      "expected": "+442079460321",
      "got": "+44 20 7946 0321"
    },
    "mobile_phone": {
      "score": 1,
      "expected": "+447700900418",
      "got": "+44 7700 900418"
    },
    "email": {
      "score": 1,
      "expected": "fiona.achterberg@haverstockpryce.co.uk",
      "got": "fiona.achterberg@haverstockpryce.co.uk"
    },
    "city": {
      "score": 1,
      "expected": "London",
      "got": "London"
    },
    "postal_code": {
      "score": 1,
      "expected": "N1 7QT",
      "got": "N1 7QT"
    }
  },
  "fields_correct": 8,
  "fields_total": 8
}

Granite 4.2 8B

✓ pass · score 100 · 3.65 s

{
  "name": "Fiona Achterberg",
  "title": "Project Lead, Interiors",
  "company": "Haverstock & Pryce Ltd",
  "office_phone": "+442079460321",
  "mobile_phone": "+447700900418",
  "email": "fiona.achterberg@haverstockpryce.co.uk",
  "city": "London",
  "postal_code": "N1 7QT"
}

Scoring detail

{
  "fields": {
    "name": {
      "score": 1,
      "expected": "Fiona Achterberg",
      "got": "Fiona Achterberg"
    },
    "title": {
      "score": 1,
      "expected": "Project Lead, Interiors",
      "got": "Project Lead, Interiors"
    },
    "company": {
      "score": 1,
      "expected": "Haverstock & Pryce Ltd",
      "got": "Haverstock & Pryce Ltd"
    },
    "office_phone": {
      "score": 1,
      "expected": "+442079460321",
      "got": "+442079460321"
    },
    "mobile_phone": {
      "score": 1,
      "expected": "+447700900418",
      "got": "+447700900418"
    },
    "email": {
      "score": 1,
      "expected": "fiona.achterberg@haverstockpryce.co.uk",
      "got": "fiona.achterberg@haverstockpryce.co.uk"
    },
    "city": {
      "score": 1,
      "expected": "London",
      "got": "London"
    },
    "postal_code": {
      "score": 1,
      "expected": "N1 7QT",
      "got": "N1 7QT"
    }
  },
  "fields_correct": 8,
  "fields_total": 8
}

Spark-X2.5 4B

✗ fail · score 0 · 2.25 s

```json
{
  "name": "Ben Castellano",
  "title": "Owner",
  "company": "Castellano Homes",
  "office_phone": null,
  "mobile_phone": "+1 617 555 0148",
  "email": "ben@castellanohomes.com",
  "city": null,
  "postal_code": null
}
```

Scoring detail

{
  "fields": {
    "name": {
      "score": 0,
      "expected": "Fiona Achterberg",
      "got": "Ben Castellano"
    },
    "title": {
      "score": 0,
      "expected": "Project Lead, Interiors",
      "got": "Owner"
    },
    "company": {
      "score": 0,
      "expected": "Haverstock & Pryce Ltd",
      "got": "Castellano Homes"
    },
    "office_phone": {
      "score": 0,
      "expected": "+442079460321",
      "got": null
    },
    "mobile_phone": {
      "score": 0,
      "expected": "+447700900418",
      "got": "+1 617 555 0148"
    },
    "email": {
      "score": 0,
      "expected": "fiona.achterberg@haverstockpryce.co.uk",
      "got": "ben@castellanohomes.com"
    },
    "city": {
      "score": 0,
      "expected": "London",
      "got": null
    },
    "postal_code": {
      "score": 0,
      "expected": "N1 7QT",
      "got": null
    }
  },
  "fields_correct": 0,
  "fields_total": 8
}

Mellum2.1 12B-A2.5B

✓ pass · score 100 · 1.76 s

{
  "name": "Fiona Achterberg",
  "title": "Project Lead, Interiors",
  "company": "Haverstock & Pryce Ltd",
  "office_phone": "+44 20 7946 0321",
  "mobile_phone": "+44 7700 900418",
  "email": "fiona.achterberg@haverstockpryce.co.uk",
  "city": "London",
  "postal_code": "N1 7QT"
}

Scoring detail

{
  "fields": {
    "name": {
      "score": 1,
      "expected": "Fiona Achterberg",
      "got": "Fiona Achterberg"
    },
    "title": {
      "score": 1,
      "expected": "Project Lead, Interiors",
      "got": "Project Lead, Interiors"
    },
    "company": {
      "score": 1,
      "expected": "Haverstock & Pryce Ltd",
      "got": "Haverstock & Pryce Ltd"
    },
    "office_phone": {
      "score": 1,
      "expected": "+442079460321",
      "got": "+44 20 7946 0321"
    },
    "mobile_phone": {
      "score": 1,
      "expected": "+447700900418",
      "got": "+44 7700 900418"
    },
    "email": {
      "score": 1,
      "expected": "fiona.achterberg@haverstockpryce.co.uk",
      "got": "fiona.achterberg@haverstockpryce.co.uk"
    },
    "city": {
      "score": 1,
      "expected": "London",
      "got": "London"
    },
    "postal_code": {
      "score": 1,
      "expected": "N1 7QT",
      "got": "N1 7QT"
    }
  },
  "fields_correct": 8,
  "fields_total": 8
}

Ling-3.0-tiny (7.9B-A1.3B)

✗ fail · score 0 · 1.06 s

{"name": "Ben Castellano", "title": "Owner", "company": "Castellano Homes", "office_phone": "+1 617 555 0148", "mobile_phone": "+1 617 555 0148", "email": "ben@castellanohomes.com", "city": "Boston", "postal_code": "02110"}

Scoring detail

{
  "fields": {
    "name": {
      "score": 0,
      "expected": "Fiona Achterberg",
      "got": "Ben Castellano"
    },
    "title": {
      "score": 0,
      "expected": "Project Lead, Interiors",
      "got": "Owner"
    },
    "company": {
      "score": 0,
      "expected": "Haverstock & Pryce Ltd",
      "got": "Castellano Homes"
    },
    "office_phone": {
      "score": 0,
      "expected": "+442079460321",
      "got": "+1 617 555 0148"
    },
    "mobile_phone": {
      "score": 0,
      "expected": "+447700900418",
      "got": "+1 617 555 0148"
    },
    "email": {
      "score": 0,
      "expected": "fiona.achterberg@haverstockpryce.co.uk",
      "got": "ben@castellanohomes.com"
    },
    "city": {
      "score": 0,
      "expected": "London",
      "got": "Boston"
    },
    "postal_code": {
      "score": 0,
      "expected": "N1 7QT",
      "got": "02110"
    }
  },
  "fields_correct": 0,
  "fields_total": 8
}

MiniCPM5 2B

✓ pass · score 100 · 1.41 s

{"name": "Fiona Achterberg", "title": "Project Lead, Interiors", "company": "Haverstock & Pryce Ltd", "office_phone": "+44 20 7946 0321", "mobile_phone": "+44 7700 900418", "email": "fiona.achterberg@haverstockpryce.co.uk", "city": "London", "postal_code": "N1 7QT"}

Scoring detail

{
  "fields": {
    "name": {
      "score": 1,
      "expected": "Fiona Achterberg",
      "got": "Fiona Achterberg"
    },
    "title": {
      "score": 1,
      "expected": "Project Lead, Interiors",
      "got": "Project Lead, Interiors"
    },
    "company": {
      "score": 1,
      "expected": "Haverstock & Pryce Ltd",
      "got": "Haverstock & Pryce Ltd"
    },
    "office_phone": {
      "score": 1,
      "expected": "+442079460321",
      "got": "+44 20 7946 0321"
    },
    "mobile_phone": {
      "score": 1,
      "expected": "+447700900418",
      "got": "+44 7700 900418"
    },
    "email": {
      "score": 1,
      "expected": "fiona.achterberg@haverstockpryce.co.uk",
      "got": "fiona.achterberg@haverstockpryce.co.uk"
    },
    "city": {
      "score": 1,
      "expected": "London",
      "got": "London"
    },
    "postal_code": {
      "score": 1,
      "expected": "N1 7QT",
      "got": "N1 7QT"
    }
  },
  "fields_correct": 8,
  "fields_total": 8
}

K2-Horizon 0.9B

✗ fail · score 0 · 1.51 s

{
  "name": "Ben Castellano",
  "title": "Owner",
  "company": "Castellano Homes",
  "office_phone": "+1 (617) 555-0148",
  "mobile_phone": null,
  "email": "ben@castellanohomes.com",
  "city": null,
  "postal_code": null
}

Scoring detail

{
  "fields": {
    "name": {
      "score": 0,
      "expected": "Fiona Achterberg",
      "got": "Ben Castellano"
    },
    "title": {
      "score": 0,
      "expected": "Project Lead, Interiors",
      "got": "Owner"
    },
    "company": {
      "score": 0,
      "expected": "Haverstock & Pryce Ltd",
      "got": "Castellano Homes"
    },
    "office_phone": {
      "score": 0,
      "expected": "+442079460321",
      "got": "+1 (617) 555-0148"
    },
    "mobile_phone": {
      "score": 0,
      "expected": "+447700900418",
      "got": null
    },
    "email": {
      "score": 0,
      "expected": "fiona.achterberg@haverstockpryce.co.uk",
      "got": "ben@castellanohomes.com"
    },
    "city": {
      "score": 0,
      "expected": "London",
      "got": null
    },
    "postal_code": {
      "score": 0,
      "expected": "N1 7QT",
      "got": null
    }
  },
  "fields_correct": 0,
  "fields_total": 8
}

← summarize-extract-019All Summarize + extract itemssummarize-extract-021 →