Unified Memory

Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread

rewrite-028

hard toneangry-to-polite

The prompt

user

Rewrite the message below so it is calm and polite, while staying clear about the problem and what the writer wants. Keep every fact (names, dates, times, amounts, reference numbers) as given, and do not add new facts or numbers. Reply with only the rewritten message, with no preamble or explanation.

"""
I am FURIOUS. This is the THIRD time I've had to explain this. My claim number is HC-2207-0419. On July 19, a tree from the city lot fell on my garage at 88 Linden Ave. Your adjuster, Gary Pruitt, came out on July 24 and told me the claim would be approved within 10 business days. It has been over a month. Nobody returns my calls, your hold times are over 45 minutes, and I've now paid $2,150 out of pocket for a tarp and emergency roof repair. If I don't get a decision by Friday, August 29, I'm filing a complaint with the state insurance department and leaving a review everywhere I can. Unbelievable.
"""
What we expected
{
  "type": "tone",
  "style": "angry-to-polite",
  "source": "I am FURIOUS. This is the THIRD time I've had to explain this. My claim number is HC-2207-0419. On July 19, a tree from the city lot fell on my garage at 88 Linden Ave. Your adjuster, Gary Pruitt, came out on July 24 and told me the claim would be approved within 10 business days. It has been over a month. Nobody returns my calls, your hold times are over 45 minutes, and I've now paid $2,150 out of pocket for a tarp and emergency roof repair. If I don't get a decision by Friday, August 29, I'm filing a complaint with the state insurance department and leaving a review everywhere I can. Unbelievable.",
  "facts": [
    {
      "name": "third time",
      "any": [
        "third",
        "3rd"
      ]
    },
    {
      "name": "claim HC-2207-0419",
      "any": [
        "hc-2207-0419"
      ]
    },
    {
      "name": "July 19",
      "any": [
        "july 19"
      ]
    },
    {
      "name": "tree on garage",
      "any": [
        "garage"
      ]
    },
    {
      "name": "88 Linden Ave",
      "any": [
        "88 linden"
      ]
    },
    {
      "name": "Gary Pruitt",
      "any": [
        "gary pruitt",
        "mr. pruitt"
      ]
    },
    {
      "name": "July 24",
      "any": [
        "july 24"
      ]
    },
    {
      "name": "10 business days",
      "any": [
        "10 business days",
        "ten business days"
      ]
    },
    {
      "name": "over a month",
      "any": [
        "month"
      ]
    },
    {
      "name": "45 minutes",
      "any": [
        "45 minutes"
      ]
    },
    {
      "name": "$2,150",
      "any": [
        "$2150"
      ]
    },
    {
      "name": "tarp",
      "any": [
        "tarp"
      ]
    },
    {
      "name": "roof",
      "any": [
        "roof"
      ]
    },
    {
      "name": "August 29",
      "any": [
        "august 29"
      ]
    },
    {
      "name": "Friday",
      "any": [
        "friday"
      ]
    },
    {
      "name": "state insurance department",
      "any": [
        "insurance department",
        "insurance regulator",
        "insurance commission"
      ]
    }
  ],
  "banned": [
    "furious",
    "unbelievable",
    "everywhere i can",
    "nobody returns"
  ],
  "banned_re": [],
  "no_caps": true,
  "caps_ok": [],
  "extra_numbers_ok": [
    "3"
  ],
  "length_ratio": [
    0.5,
    3
  ],
  "reference": "Hello,\n\nI'm writing to follow up on my claim, number HC-2207-0419. This is the third time I've reached out, so I'd be grateful for your help getting it resolved.\n\nOn July 19, a tree from the city lot fell on my garage at 88 Linden Ave. Your adjuster, Gary Pruitt, inspected the damage on July 24 and told me the claim would be approved within 10 business days. It has now been over a month. I haven't been able to reach anyone by phone, and hold times have been over 45 minutes. In the meantime, I've paid $2,150 out of pocket for a tarp and an emergency roof repair.\n\nCould you please give me a decision by Friday, August 29? If I don't hear back by then, I will need to file a complaint with the state insurance department.\n\nThank you for your attention to this."
}

What each model answered

MiMo-V2.6 Distill 9B

✗ fail · score 89 (scripted checks 97, judges 8.0/10 from 3) · 6.40 s

Dear Claims Team,

I hope you are well. I am writing regarding claim number HC-2207-0419.

On July 19, a tree from the city lot fell on my garage at 88 Linden Ave. Your adjuster, Gary Pruitt, visited on July 24 and told me the claim would be approved within 10 business days. It has now been over a month since that conversation.

I have tried several times to reach someone, but my calls are not being returned, and hold times have exceeded 45 minutes. In the meantime, I have paid $2,150 out of pocket for a tarp and emergency roof repair.

I would appreciate a decision by Friday, August 29. If I do not receive one by then, I will need to file a complaint with the state insurance department and leave a review on the channels available to me.

Thank you for your attention to this matter.

Sincerely,
F. Pruitt

Judge medians: Tone match 10 · Faithfulness 5 · Naturalness 9

Scoring detail
{
  "cleaned_words": 151,
  "similarity_to_source": 0.586,
  "facts_missing": [
    "third time"
  ],
  "banned_hits": [],
  "new_numbers": [],
  "length_ratio": 1.313,
  "length_bounds": [
    0.5,
    3
  ],
  "gate": 1,
  "checks": {
    "facts": 0.9375,
    "banned": 1,
    "numbers": 1,
    "changed": 1,
    "length": 1
  }
}

Ornith-1.5 9B

✗ fail · score 79 (scripted checks 77, judges 8.0/10 from 3) · 5.25 s

Dear [Recipient],

I am writing regarding my claim, number HC-2207-0419. On July 19, a tree from the city lot fell on my garage at 88 Linden Ave. Your adjuster, Gary Pruitt, came out on July 24 and told me the claim would be approved within 10 business days. It has now been over a month, and I have not yet received a decision.

I have tried several times to reach you by phone, but I have not been able to get through, as your hold times are over 45 minutes. In the meantime, I have had to pay $2,150 out of pocket for a tarp and emergency roof repair.

I would be very grateful if you could provide a decision by Friday, August 29. If I do not receive one by then, I will have to file a complaint with the state insurance department and leave a review everywhere I can.

Thank you for your attention to this matter.

Sincerely,
[Your Name]

Judge medians: Tone match 9 · Faithfulness 7 · Naturalness 8

Scoring detail
{
  "cleaned_words": 162,
  "similarity_to_source": 0.606,
  "facts_missing": [
    "third time"
  ],
  "banned_hits": [
    "everywhere i can"
  ],
  "new_numbers": [],
  "length_ratio": 1.409,
  "length_bounds": [
    0.5,
    3
  ],
  "gate": 1,
  "checks": {
    "facts": 0.9375,
    "banned": 0,
    "numbers": 1,
    "changed": 1,
    "length": 1
  }
}

Granite 4.2 8B

✗ fail · score 90 (scripted checks 97, judges 8.3/10 from 3) · 6.50 s

Hello,

I hope you are well. My name is the claim holder for claim number HC-2207-0419. On July 19, a tree from the city lot fell on my garage at 88 Linden Ave. On July 24, your adjuster Gary Pruitt visited the site and indicated that the claim would be approved within 10 business days. It has now been more than a month since that estimate, and I have not yet received a decision.

In the meantime, I have incurred $2,150 out‑of‑pocket for a tarp and emergency roof repair. I have also experienced long hold times (over 45 minutes) and have not yet received a return call.

Could you please provide an update on the status of my claim and let me know when I can expect a decision? I would appreciate a response by Friday, August 29. If I do not receive a decision by that date, I will need to file a complaint with the state insurance department and leave a review of my experience.

Thank you for your attention to this matter. I look forward to your prompt response.

Sincerely,  
[Your name]

Judge medians: Tone match 10 · Faithfulness 8 · Naturalness 7

Scoring detail
{
  "cleaned_words": 185,
  "similarity_to_source": 0.427,
  "facts_missing": [
    "third time"
  ],
  "banned_hits": [],
  "new_numbers": [],
  "length_ratio": 1.609,
  "length_bounds": [
    0.5,
    3
  ],
  "gate": 1,
  "checks": {
    "facts": 0.9375,
    "banned": 1,
    "numbers": 1,
    "changed": 1,
    "length": 1
  }
}

Spark-X2.5 4B

✗ fail · score 89 (scripted checks 97, judges 8.0/10 from 3) · 3.16 s

I am writing to express my frustration regarding the delay in processing my claim. My claim number is HC-2207-0419. On July 19, a tree fell on my garage at 88 Linden Ave. Your adjuster, Gary Pruitt, visited on July 24 and informed me that the claim would be approved within 10 business days. However, it has been over a month, and I have not received a decision. I have also been unable to reach you, as hold times exceed 45 minutes. Additionally, I have already paid $2,150 out of pocket for a tarp and emergency roof repair. If I do not receive a decision by Friday, August 29, I will have to file a complaint with the state insurance department and leave reviews where possible.

Judge medians: Tone match 9 · Faithfulness 6 · Naturalness 9

Scoring detail
{
  "cleaned_words": 125,
  "similarity_to_source": 0.625,
  "facts_missing": [
    "third time"
  ],
  "banned_hits": [],
  "new_numbers": [],
  "length_ratio": 1.087,
  "length_bounds": [
    0.5,
    3
  ],
  "gate": 1,
  "checks": {
    "facts": 0.9375,
    "banned": 1,
    "numbers": 1,
    "changed": 1,
    "length": 1
  }
}

Mellum2.1 12B-A2.5B

✗ fail · score 73 (scripted checks 80, judges 6.7/10 from 3) · 2.17 s

I am upset. This is the third time I have had to explain this. My claim number is HC-2207-0419. On July 19, a tree from the city lot fell on my garage at 88 Linden Ave. Your adjuster, Gary Pruitt, came out on July 24 and said the claim would be approved within 10 business days. It has been more than a month. No one returns my calls; your hold times are over 45 minutes, and I have now paid $2,150 out of pocket for a tarp and emergency roof repair. If I do not receive a decision by Friday, August 29, I will file a complaint with the state insurance department and leave a review wherever I can. This is unbelievable.

Judge medians: Tone match 4 · Faithfulness 10 · Naturalness 6

Scoring detail
{
  "cleaned_words": 122,
  "similarity_to_source": 0.852,
  "facts_missing": [],
  "banned_hits": [
    "unbelievable"
  ],
  "new_numbers": [],
  "length_ratio": 1.061,
  "length_bounds": [
    0.5,
    3
  ],
  "gate": 1,
  "checks": {
    "facts": 1,
    "banned": 0,
    "numbers": 1,
    "changed": 1,
    "length": 1
  }
}

Ling-3.0-tiny (7.9B-A1.3B)

✓ pass · score 87 (scripted checks 100, judges 7.3/10 from 3) · 1.58 s

I am very frustrated. This is the third time I have had to explain this. My claim number is HC-2207-0419. On July 19, a tree from the city lot fell on my garage at 88 Linden Ave. Your adjuster, Gary Pruitt, came out on July 24 and told me the claim would be approved within 10 business days. It has been over a month. I am not receiving any calls, your hold times are over 45 minutes, and I have now paid $2,150 out of pocket for a tarp and emergency roof repair. If I do not receive a decision by Friday, August 29, I will file a complaint with the state insurance department and leave a review wherever I can.

Judge medians: Tone match 6 · Faithfulness 8 · Naturalness 8

Scoring detail
{
  "cleaned_words": 121,
  "similarity_to_source": 0.864,
  "facts_missing": [],
  "banned_hits": [],
  "new_numbers": [],
  "length_ratio": 1.052,
  "length_bounds": [
    0.5,
    3
  ],
  "gate": 1,
  "checks": {
    "facts": 1,
    "banned": 1,
    "numbers": 1,
    "changed": 1,
    "length": 1
  }
}

MiniCPM5 2B

✗ fail · score 74 (scripted checks 77, judges 7.0/10 from 3) · 2.15 s

I am writing to express concern regarding claim HC-2207-0419. On July 19, a tree from the city lot fell on my garage at 88 Linden Ave. My adjuster, Gary Pruitt, stated that the claim would be approved within 10 business days on July 24. It has been over a month since then. I have not received responses to my calls, and hold times have exceeded 45 minutes. I have paid $2,150 out of pocket for a tarp and emergency roof repair. If I do not receive a decision by Friday, August 29, I will file a complaint with the state insurance department and leave a review everywhere I can. I find this situation unbelievable.

Judge medians: Tone match 7 · Faithfulness 7 · Naturalness 7

Scoring detail
{
  "cleaned_words": 114,
  "similarity_to_source": 0.672,
  "facts_missing": [
    "third time"
  ],
  "banned_hits": [
    "unbelievable",
    "everywhere i can"
  ],
  "new_numbers": [],
  "length_ratio": 0.991,
  "length_bounds": [
    0.5,
    3
  ],
  "gate": 1,
  "checks": {
    "facts": 0.9375,
    "banned": 0,
    "numbers": 1,
    "changed": 1,
    "length": 1
  }
}

K2-Horizon 0.9B

✗ fail · score 48 (scripted checks 35, judges 6.0/10 from 3) · 0.85 s

I am FURIOUS. This is the THIRD time I've had to explain this. My claim number is HC-2207-0419. On July 19, a tree from the city lot fell on my garage at 88 Linden Ave. Your adjuster, Gary Pruitt, came out on July 24 and told me the claim would be approved within 10 business days. It has been over a month. Nobody returns my calls, your hold times are over 45 minutes, and I've now paid $2,150 out of pocket for a tarp and emergency roof repair. If I don't get a decision by Friday, August 29, I'm filing a complaint with the state insurance department and leaving a review everywhere I can.

Judge medians: Tone match 0 · Faithfulness 10 · Naturalness 8

Scoring detail
{
  "cleaned_words": 114,
  "similarity_to_source": 0.996,
  "facts_missing": [],
  "banned_hits": [
    "furious",
    "everywhere i can",
    "nobody returns",
    "ALLCAPS:FURIOUS,THIRD"
  ],
  "new_numbers": [],
  "length_ratio": 0.991,
  "length_bounds": [
    0.5,
    3
  ],
  "gate": 0.5,
  "checks": {
    "facts": 1,
    "banned": 0,
    "numbers": 1,
    "changed": 0,
    "length": 1
  }
}

← rewrite-027All Rewrite and proofread itemsrewrite-029 →