Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract

summarize-extract-032

medium summarizationemail

The prompt

user

Summarize the family email thread (newest message first) below in at most 70 words. Say what was finally decided and what people need to do. Reply with only the summary.

"""
From: Joanne Pruitt
Date: Sun, May 10, 2026, 8:12 PM
Subject: Re: Re: Re: Beach week 2026!!

OK it's settled! I just put down the $500 deposit on the Gull Point cottage for July 11-18 (Saturday to Saturday). Total is $2,450 including cleaning fee and tax. Split four ways that's $612.50 per family. Please Venmo or check me your share by June 1 so I can pay the balance on time.

Ray called - he can't come this year because of the knee surgery, so it's just the four of us families. The cottage sleeps 12 so we're fine.

Love, Mom

> From: Gina Pruitt-Lowe
> Date: Sat, May 9, 2026, 11:30 AM
> I vote Gull Point. The Sandpiper Inn would be $3,100 for the same week AND no kitchen. Also July 4 week is booked solid everywhere.
>
> > From: Marcus Pruitt
> > Date: Fri, May 8, 2026, 6:02 PM
> > Our options as far as I can tell:
> > 1) Sandpiper Inn, 4 rooms, July 4-11, about $3,100
> > 2) Gull Point cottage (sleeps 12, full kitchen, 2 blocks from beach), July 11-18, $2,450 all in
> > 3) Skip the beach and do the lake cabin again Aug 1-8, $1,800
> > Kids want the beach. Ray wants the lake because of stairs.
> >
> > > From: Joanne Pruitt
> > > Date: Thu, May 7, 2026, 9:45 AM
> > > Time to plan beach week! Who's in and which week works? Ray, Gina, Marcus, Tim - let me know.
"""

{
  "kind": "summarize",
  "word_limit": 70,
  "facts": [
    {
      "id": "choice",
      "desc": "they booked the Gull Point cottage",
      "any": [
        "gull point"
      ]
    },
    {
      "id": "dates",
      "desc": "for July 11-18",
      "any": [
        "july\\s*11|7/11|11(th)?\\s*(-|to|through|–)\\s*(july\\s*)?18"
      ]
    },
    {
      "id": "cost",
      "desc": "total $2,450 split four ways ($612.50 per family)",
      "any": [
        "2,?450|612(\\.50)?|four ways|split (four|4)|per family"
      ]
    },
    {
      "id": "pay_by",
      "desc": "each family pays their share by June 1",
      "any": [
        "(\\bjune?\\s*1(st|nd|rd|th)?\\b|\\b1(st|nd|rd|th)?\\s+(of\\s+)?june?\\b|\\b0?6/0?1\\b|\\b2026-06-01\\b|\\bfirst\\s+(of\\s+)?june?\\b|\\bjune?\\s+(the\\s+)?first\\b)"
      ]
    },
    {
      "id": "ray",
      "desc": "Ray can't come (knee surgery)",
      "any": [
        "\\bray\\b[^.;]{0,50}(can'?t|cannot|not|won'?t|unable|surgery|knee|skip|out)"
      ]
    }
  ],
  "allowed_numbers": [],
  "source_words": 262,
  "reference_summary": "Final plan: Joanne booked the Gull Point cottage for July 11-18 and paid the $500 deposit. The $2,450 total is split four ways, $612.50 per family, which each family should send Joanne by June 1. Ray can't come because of his knee surgery."
}

What each model answered

Decided: Gull Point cottage, July 11–18, 2026, $2,450 total ($612.50 per family). Ray can't attend due to knee surgery. Each family must Venmo or check Joanne by June 1.

{
  "facts": {
    "choice": true,
    "dates": true,
    "cost": true,
    "pay_by": true,
    "ray": true
  },
  "facts_frac": 1,
  "word_count": 29,
  "word_limit": 70,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

The family decided on the Gull Point cottage for July 11–18, 2026, at $2,450 total ($612.50 per family). Ray can't attend due to knee surgery, leaving four families. Everyone must Venmo or check Joanne their share by June 1 so she can pay the balance.

{
  "facts": {
    "choice": true,
    "dates": true,
    "cost": true,
    "pay_by": true,
    "ray": true
  },
  "facts_frac": 1,
  "word_count": 45,
  "word_limit": 70,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

The family finally decided to rent the Gull Point cottage for July 11‑18, paying the $2,450 total (including fees and tax). Each of the four families owes $612.50. Ray cannot attend due to knee surgery, so only the four families go. They need to send their $612.50 share via Venmo or check by June 1 so Joanne can pay the balance on time.

{
  "facts": {
    "choice": true,
    "dates": true,
    "cost": true,
    "pay_by": true,
    "ray": true
  },
  "facts_frac": 1,
  "word_count": 63,
  "word_limit": 70,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

The family has finally decided to book the Gull Point cottage for July 11-18. Joanne will collect the $612.50 share via Venmo or check by June 1. Ray cannot attend due to knee surgery, leaving the four families to manage the 12-person cottage.

{
  "facts": {
    "choice": true,
    "dates": true,
    "cost": true,
    "pay_by": true,
    "ray": true
  },
  "facts_frac": 1,
  "word_count": 43,
  "word_limit": 70,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

The family decided on the Gull Point cottage for July 11‑18 (Saturday to Saturday) after comparing options; the Sandpiper Inn was deemed too expensive and lacked a kitchen, while the lake cabin was not desired. The total cost is $2,450 including cleaning fee and tax, split four ways at $612.50 per family. Everyone must pay their share by June 1 via Venmo or check to allow the balance payment.

{
  "facts": {
    "choice": true,
    "dates": true,
    "cost": true,
    "pay_by": true,
    "ray": false
  },
  "facts_frac": 0.8,
  "word_count": 69,
  "word_limit": 70,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

The family decided to book the Gull Point cottage for July 11-18, with a $500 deposit and $612.50 per family. Ray is excluded due to knee surgery, and the four families will split the cost. The lake cabin is a backup option if needed.

{
  "facts": {
    "choice": true,
    "dates": true,
    "cost": true,
    "pay_by": false,
    "ray": true
  },
  "facts_frac": 0.8,
  "word_count": 44,
  "word_limit": 70,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

The $2,450 deposit for the Gull Point cottage (July 11-18) is settled, with a $612.50 share per family. Venmo or check by June 1 to pay the balance. Ray can't attend due to knee surgery, so only four families are going.

{
  "facts": {
    "choice": true,
    "dates": true,
    "cost": true,
    "pay_by": true,
    "ray": true
  },
  "facts_frac": 1,
  "word_count": 41,
  "word_limit": 70,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

The family decided on Gull Point cottage for July 11-18, $2,450 total, split $612.50 per family. Joanne Pruitt needs to deposit $500 by June 1 to cover the balance.

{
  "facts": {
    "choice": true,
    "dates": true,
    "cost": true,
    "pay_by": true,
    "ray": false
  },
  "facts_frac": 0.8,
  "word_count": 29,
  "word_limit": 70,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

← summarize-extract-031All Summarize + extract itemssummarize-extract-033 →