Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract

summarize-extract-016

medium extractionshipping

The prompt

user

Extract information from the email below. Reply with only a JSON object with exactly these keys:
- "order_number" (string): order number
- "carrier" (string): carrier
- "tracking_number" (string): tracking number
- "original_delivery_date" (string "YYYY-MM-DD"): originally promised delivery date
- "new_delivery_date" (string "YYYY-MM-DD"): new estimated delivery date
- "items" (array): array of {"name": string, "quantity": number}
- "account_credit" (number): credit added to the account

Rules: include every key listed. Use null for any field that is not stated in the text (do not guess). Dates are YYYY-MM-DD. Times are HH:MM in 24-hour format. Amounts and counts are plain JSON numbers (no currency symbols, no commas). Phone numbers are in international format with + and the country code, for example +1 312 555 0100; leave out any national trunk prefix 0, so (0)20 or 020 becomes 20 after +44.

Email:
"""
From: Summit & Sage Outfitters <orders@summitsage.shop>
Date: Mon, 2 Mar 2026 07:30
Subject: Update on order SS-448120: new delivery date

Hi Alex,

We're sorry - your package is running a little behind. A weather delay at our carrier's hub means it won't make the original Tuesday, March 3 delivery date. The new estimated delivery is Friday, March 6.

Carrier: TrailPost Express
Tracking: TPX 0098 3321 7745

What's in your box:
- Trail Runner Socks (L), qty 2
- Merino Base Layer Top - Charcoal - M, qty 1
- Collapsible Water Bottle 750ml, qty 1

As a thank-you for your patience, we've added a $10 credit to your account.
"""

What we expected

{
  "kind": "extract",
  "answer": {
    "order_number": "SS-448120",
    "carrier": "TrailPost Express",
    "tracking_number": "TPX 0098 3321 7745",
    "original_delivery_date": "2026-03-03",
    "new_delivery_date": "2026-03-06",
    "items": [
      {
        "name": "Trail Runner Socks (L)",
        "quantity": 2
      },
      {
        "name": "Merino Base Layer Top - Charcoal - M",
        "quantity": 1
      },
      {
        "name": "Collapsible Water Bottle 750ml",
        "quantity": 1
      }
    ],
    "account_credit": 10
  },
  "fields": {
    "order_number": {
      "type": "code"
    },
    "carrier": {
      "type": "string",
      "accept": [
        "TrailPost"
      ]
    },
    "tracking_number": {
      "type": "code"
    },
    "original_delivery_date": {
      "type": "date"
    },
    "new_delivery_date": {
      "type": "date"
    },
    "items": {
      "type": "list",
      "item_fields": {
        "name": {
          "type": "text",
          "needles_map": {
            "Trail Runner Socks (L)": [
              "sock"
            ],
            "Merino Base Layer Top - Charcoal - M": [
              "merino",
              "base layer"
            ],
            "Collapsible Water Bottle 750ml": [
              "bottle"
            ]
          }
        },
        "quantity": {
          "type": "number"
        }
      }
    },
    "account_credit": {
      "type": "number"
    }
  }
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 4.74 s

{"order_number": "SS-448120", "carrier": "TrailPost Express", "tracking_number": "TPX 0098 3321 7745", "original_delivery_date": "2026-03-03", "new_delivery_date": "2026-03-06", "items": [{"name": "Trail Runner Socks (L)", "quantity": 2}, {"name": "Merino Base Layer Top - Charcoal - M", "quantity": 1}, {"name": "Collapsible Water Bottle 750ml", "quantity": 1}], "account_credit": 10}

Scoring detail

{
  "fields": {
    "order_number": {
      "score": 1,
      "expected": "SS-448120",
      "got": "SS-448120"
    },
    "carrier": {
      "score": 1,
      "expected": "TrailPost Express",
      "got": "TrailPost Express"
    },
    "tracking_number": {
      "score": 1,
      "expected": "TPX 0098 3321 7745",
      "got": "TPX 0098 3321 7745"
    },
    "original_delivery_date": {
      "score": 1,
      "expected": "2026-03-03",
      "got": "2026-03-03"
    },
    "new_delivery_date": {
      "score": 1,
      "expected": "2026-03-06",
      "got": "2026-03-06"
    },
    "items": {
      "score": 1,
      "expected": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ],
      "got": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ]
    },
    "account_credit": {
      "score": 1,
      "expected": 10,
      "got": 10
    }
  },
  "fields_correct": 7,
  "fields_total": 7
}

Ornith-1.5 9B

✓ pass · score 100 · 4.99 s

{
  "order_number": "SS-448120",
  "carrier": "TrailPost Express",
  "tracking_number": "TPX 0098 3321 7745",
  "original_delivery_date": "2026-03-03",
  "new_delivery_date": "2026-03-06",
  "items": [
    {"name": "Trail Runner Socks (L)", "quantity": 2},
    {"name": "Merino Base Layer Top - Charcoal - M", "quantity": 1},
    {"name": "Collapsible Water Bottle 750ml", "quantity": 1}
  ],
  "account_credit": 10
}

Scoring detail

{
  "fields": {
    "order_number": {
      "score": 1,
      "expected": "SS-448120",
      "got": "SS-448120"
    },
    "carrier": {
      "score": 1,
      "expected": "TrailPost Express",
      "got": "TrailPost Express"
    },
    "tracking_number": {
      "score": 1,
      "expected": "TPX 0098 3321 7745",
      "got": "TPX 0098 3321 7745"
    },
    "original_delivery_date": {
      "score": 1,
      "expected": "2026-03-03",
      "got": "2026-03-03"
    },
    "new_delivery_date": {
      "score": 1,
      "expected": "2026-03-06",
      "got": "2026-03-06"
    },
    "items": {
      "score": 1,
      "expected": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ],
      "got": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ]
    },
    "account_credit": {
      "score": 1,
      "expected": 10,
      "got": 10
    }
  },
  "fields_correct": 7,
  "fields_total": 7
}

Granite 4.2 8B

✗ fail · score 86 · 5.08 s

{
  "order_number": "SS-448120",
  "carrier": "TrailPost Express",
  "tracking_number": "TPX0098333217745",
  "original_delivery_date": "2026-03-03",
  "new_delivery_date": "2026-03-06",
  "items": [
    {
      "name": "Trail Runner Socks (L)",
      "quantity": 2
    },
    {
      "name": "Merino Base Layer Top - Charcoal - M",
      "quantity": 1
    },
    {
      "name": "Collapsible Water Bottle 750ml",
      "quantity": 1
    }
  ],
  "account_credit": 10
}

Scoring detail

{
  "fields": {
    "order_number": {
      "score": 1,
      "expected": "SS-448120",
      "got": "SS-448120"
    },
    "carrier": {
      "score": 1,
      "expected": "TrailPost Express",
      "got": "TrailPost Express"
    },
    "tracking_number": {
      "score": 0,
      "expected": "TPX 0098 3321 7745",
      "got": "TPX0098333217745"
    },
    "original_delivery_date": {
      "score": 1,
      "expected": "2026-03-03",
      "got": "2026-03-03"
    },
    "new_delivery_date": {
      "score": 1,
      "expected": "2026-03-06",
      "got": "2026-03-06"
    },
    "items": {
      "score": 1,
      "expected": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ],
      "got": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ]
    },
    "account_credit": {
      "score": 1,
      "expected": 10,
      "got": 10
    }
  },
  "fields_correct": 6,
  "fields_total": 7
}

Spark-X2.5 4B

✓ pass · score 100 · 3.89 s

{
  "order_number": "SS-448120",
  "carrier": "TrailPost Express",
  "tracking_number": "TPX 0098 3321 7745",
  "original_delivery_date": "2026-03-03",
  "new_delivery_date": "2026-03-06",
  "items": [
    {
      "name": "Trail Runner Socks (L)",
      "quantity": 2
    },
    {
      "name": "Merino Base Layer Top - Charcoal - M",
      "quantity": 1
    },
    {
      "name": "Collapsible Water Bottle 750ml",
      "quantity": 1
    }
  ],
  "account_credit": 10
}

Scoring detail

{
  "fields": {
    "order_number": {
      "score": 1,
      "expected": "SS-448120",
      "got": "SS-448120"
    },
    "carrier": {
      "score": 1,
      "expected": "TrailPost Express",
      "got": "TrailPost Express"
    },
    "tracking_number": {
      "score": 1,
      "expected": "TPX 0098 3321 7745",
      "got": "TPX 0098 3321 7745"
    },
    "original_delivery_date": {
      "score": 1,
      "expected": "2026-03-03",
      "got": "2026-03-03"
    },
    "new_delivery_date": {
      "score": 1,
      "expected": "2026-03-06",
      "got": "2026-03-06"
    },
    "items": {
      "score": 1,
      "expected": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ],
      "got": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ]
    },
    "account_credit": {
      "score": 1,
      "expected": 10,
      "got": 10
    }
  },
  "fields_correct": 7,
  "fields_total": 7
}

Mellum2.1 12B-A2.5B

✓ pass · score 100 · 2.16 s

{
  "order_number": "SS-448120",
  "carrier": "TrailPost Express",
  "tracking_number": "TPX 0098 3321 7745",
  "original_delivery_date": "2026-03-03",
  "new_delivery_date": "2026-03-06",
  "items": [
    {"name": "Trail Runner Socks (L)", "quantity": 2},
    {"name": "Merino Base Layer Top - Charcoal - M", "quantity": 1},
    {"name": "Collapsible Water Bottle 750ml", "quantity": 1}
  ],
  "account_credit": 10
}

Scoring detail

{
  "fields": {
    "order_number": {
      "score": 1,
      "expected": "SS-448120",
      "got": "SS-448120"
    },
    "carrier": {
      "score": 1,
      "expected": "TrailPost Express",
      "got": "TrailPost Express"
    },
    "tracking_number": {
      "score": 1,
      "expected": "TPX 0098 3321 7745",
      "got": "TPX 0098 3321 7745"
    },
    "original_delivery_date": {
      "score": 1,
      "expected": "2026-03-03",
      "got": "2026-03-03"
    },
    "new_delivery_date": {
      "score": 1,
      "expected": "2026-03-06",
      "got": "2026-03-06"
    },
    "items": {
      "score": 1,
      "expected": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ],
      "got": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ]
    },
    "account_credit": {
      "score": 1,
      "expected": 10,
      "got": 10
    }
  },
  "fields_correct": 7,
  "fields_total": 7
}

Ling-3.0-tiny (7.9B-A1.3B)

✓ pass · score 100 · 1.55 s

{"order_number": "SS-448120", "carrier": "TrailPost Express", "tracking_number": "TPX 0098 3321 7745", "original_delivery_date": "2026-03-03", "new_delivery_date": "2026-03-06", "items": [{"name": "Trail Runner Socks (L)", "quantity": 2}, {"name": "Merino Base Layer Top - Charcoal - M", "quantity": 1}, {"name": "Collapsible Water Bottle 750ml", "quantity": 1}], "account_credit": 10}

Scoring detail

{
  "fields": {
    "order_number": {
      "score": 1,
      "expected": "SS-448120",
      "got": "SS-448120"
    },
    "carrier": {
      "score": 1,
      "expected": "TrailPost Express",
      "got": "TrailPost Express"
    },
    "tracking_number": {
      "score": 1,
      "expected": "TPX 0098 3321 7745",
      "got": "TPX 0098 3321 7745"
    },
    "original_delivery_date": {
      "score": 1,
      "expected": "2026-03-03",
      "got": "2026-03-03"
    },
    "new_delivery_date": {
      "score": 1,
      "expected": "2026-03-06",
      "got": "2026-03-06"
    },
    "items": {
      "score": 1,
      "expected": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ],
      "got": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ]
    },
    "account_credit": {
      "score": 1,
      "expected": 10,
      "got": 10
    }
  },
  "fields_correct": 7,
  "fields_total": 7
}

MiniCPM5 2B

✓ pass · score 100 · 1.53 s

{"order_number":"SS-448120","carrier":"TrailPost Express","tracking_number":"TPX 0098 3321 7745","original_delivery_date":"2026-03-03","new_delivery_date":"2026-03-06","items":[{"name":"Trail Runner Socks (L)","quantity":2},{"name":"Merino Base Layer Top - Charcoal - M","quantity":1},{"name":"Collapsible Water Bottle 750ml","quantity":1}],"account_credit":10}

Scoring detail

{
  "fields": {
    "order_number": {
      "score": 1,
      "expected": "SS-448120",
      "got": "SS-448120"
    },
    "carrier": {
      "score": 1,
      "expected": "TrailPost Express",
      "got": "TrailPost Express"
    },
    "tracking_number": {
      "score": 1,
      "expected": "TPX 0098 3321 7745",
      "got": "TPX 0098 3321 7745"
    },
    "original_delivery_date": {
      "score": 1,
      "expected": "2026-03-03",
      "got": "2026-03-03"
    },
    "new_delivery_date": {
      "score": 1,
      "expected": "2026-03-06",
      "got": "2026-03-06"
    },
    "items": {
      "score": 1,
      "expected": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ],
      "got": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ]
    },
    "account_credit": {
      "score": 1,
      "expected": 10,
      "got": 10
    }
  },
  "fields_correct": 7,
  "fields_total": 7
}

K2-Horizon 0.9B

✓ pass · score 100 · 0.88 s

{
  "order_number": "SS-448120",
  "carrier": "TrailPost Express",
  "tracking_number": "TPX 0098 3321 7745",
  "original_delivery_date": "2026-03-03",
  "new_delivery_date": "2026-03-06",
  "items": [
    {"name": "Trail Runner Socks (L)", "quantity": 2},
    {"name": "Merino Base Layer Top - Charcoal - M", "quantity": 1},
    {"name": "Collapsible Water Bottle 750ml", "quantity": 1}
  ],
  "account_credit": 10
}

Scoring detail

{
  "fields": {
    "order_number": {
      "score": 1,
      "expected": "SS-448120",
      "got": "SS-448120"
    },
    "carrier": {
      "score": 1,
      "expected": "TrailPost Express",
      "got": "TrailPost Express"
    },
    "tracking_number": {
      "score": 1,
      "expected": "TPX 0098 3321 7745",
      "got": "TPX 0098 3321 7745"
    },
    "original_delivery_date": {
      "score": 1,
      "expected": "2026-03-03",
      "got": "2026-03-03"
    },
    "new_delivery_date": {
      "score": 1,
      "expected": "2026-03-06",
      "got": "2026-03-06"
    },
    "items": {
      "score": 1,
      "expected": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ],
      "got": [
        {
          "name": "Trail Runner Socks (L)",
          "quantity": 2
        },
        {
          "name": "Merino Base Layer Top - Charcoal - M",
          "quantity": 1
        },
        {
          "name": "Collapsible Water Bottle 750ml",
          "quantity": 1
        }
      ]
    },
    "account_credit": {
      "score": 1,
      "expected": 10,
      "got": 10
    }
  },
  "fields_correct": 7,
  "fields_total": 7
}

← summarize-extract-015All Summarize + extract itemssummarize-extract-017 →