Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread
rewrite-031
hard toneangry-to-polite
The prompt
user
Rewrite the message below so it is calm and polite, while staying clear about the problem and what the writer wants. Keep every fact (names, dates, times, amounts, reference numbers) as given, and do not add new facts or numbers. Reply with only the rewritten message, with no preamble or explanation. """ WORST anniversary dinner EVER. We booked a table for 2 at 7:30 under 'Okonkwo' three weeks ago and still waited 50 minutes. The waiter (Brent?) forgot our appetizer, my husband's steak came out raw, and then you charged us $214 including an 'automatic' 20% tip for a party of TWO?!? Never coming back. Zero stars if I could. """
What we expected
{
"type": "tone",
"style": "angry-to-polite",
"source": "WORST anniversary dinner EVER. We booked a table for 2 at 7:30 under 'Okonkwo' three weeks ago and still waited 50 minutes. The waiter (Brent?) forgot our appetizer, my husband's steak came out raw, and then you charged us $214 including an 'automatic' 20% tip for a party of TWO?!? Never coming back. Zero stars if I could.",
"facts": [
{
"name": "anniversary",
"any": [
"anniversary"
]
},
{
"name": "table for 2",
"any": [
"2",
"two"
]
},
{
"name": "7:30",
"any": [
"7:30"
]
},
{
"name": "Okonkwo",
"any": [
"okonkwo"
]
},
{
"name": "booked three weeks ahead",
"any": [
"three weeks",
"3 weeks"
]
},
{
"name": "waited 50 minutes",
"any": [
"50 minutes"
]
},
{
"name": "Brent",
"any": [
"brent"
]
},
{
"name": "appetizer",
"any": [
"appetizer"
]
},
{
"name": "steak raw",
"any": [
"raw",
"undercooked"
]
},
{
"name": "$214",
"any": [
"$214"
]
},
{
"name": "20% tip",
"any": [
"20%",
"20 percent"
]
}
],
"banned": [
"worst",
"never coming back",
"zero stars",
"?!"
],
"banned_re": [],
"no_caps": true,
"caps_ok": [],
"extra_numbers_ok": [
"3"
],
"length_ratio": [
0.5,
3
],
"reference": "Hello,\n\nI wanted to share some feedback about our anniversary dinner. We booked a table for two at 7:30 under the name Okonkwo three weeks in advance, but we still waited 50 minutes to be seated. Our server, Brent, I believe, forgot our appetizer, and my husband's steak came out raw. We were also surprised to be charged $214, including an automatic 20% tip, for a party of two.\n\nWe were really disappointed, since this was a special occasion. I hope this feedback is helpful.\n\nSincerely,"
}What each model answered
MiMo-V2.6 Distill 9B
✗ fail · score 90 (scripted checks 96, judges 8.3/10 from 3) · 4.05 s
I wanted to share feedback about our recent anniversary dinner, as we are disappointed with the experience. We booked a table for two at 7:30 under the name "Okonkwo" three weeks ago, and we waited 50 minutes. Our appetizer was not served, and my husband's steak came out raw. We were also charged $214, including what we were told was an automatic 20% tip, for a party of two. We would appreciate it if you could review this and let us know how you would like to resolve it.
Judge medians: Tone match 10 · Faithfulness 5 · Naturalness 10
Scoring detail
{
"cleaned_words": 89,
"similarity_to_source": 0.408,
"facts_missing": [
"Brent"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.534,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 0.9091,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Ornith-1.5 9B
✓ pass · score 90 (scripted checks 100, judges 8.0/10 from 3) · 4.17 s
Dear Sir or Madam, I am writing to share my disappointment with you regarding our recent anniversary dinner, which unfortunately did not meet our expectations. We booked a table for two at 7:30 under the name 'Okonkwo' three weeks ago, and we still waited 50 minutes to be seated. During the meal, the waiter (I believe his name was Brent) forgot to bring our appetizer, and my husband's steak came out raw. Additionally, we were charged $214, which included an automatic 20% tip for a party of two. I would be grateful if you could review this situation and advise how you would like to resolve it. We had hoped for a memorable anniversary celebration and would appreciate your attention to this matter. Thank you for your time and understanding. Kind regards,
Judge medians: Tone match 10 · Faithfulness 6 · Naturalness 8
Scoring detail
{
"cleaned_words": 132,
"similarity_to_source": 0.368,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 2.276,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Granite 4.2 8B
✗ fail · score 84 (scripted checks 92, judges 7.7/10 from 3) · 4.38 s
Hello, I am writing to share some concerns about our recent dinner at your restaurant under the name 'Okonkwo.' We booked a table for two for 7:30 pm three weeks ago, and we waited about 50 minutes before being seated. During the meal, the waiter (possibly named Brent) forgot our appetizer, and my husband’s steak arrived raw. Additionally, the final bill showed a charge of $214, which included what appeared to be an automatic 20 % tip for a party of two. Could you please clarify how the tip was applied and address the issues with the appetizer and the steak? I would appreciate any assistance you can provide to resolve these concerns. Thank you for your attention to this matter. Sincerely, [Your name]
Judge medians: Tone match 10 · Faithfulness 5 · Naturalness 8
Scoring detail
{
"cleaned_words": 123,
"similarity_to_source": 0.297,
"facts_missing": [
"anniversary",
"20% tip"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 2.121,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 0.8182,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Spark-X2.5 4B
✓ pass · score 87 (scripted checks 100, judges 7.3/10 from 3) · 2.15 s
I am writing to express my very serious and disappointed concerns regarding our recent anniversary dinner. We booked a table for two at 7:30 at 'Okonkwo' three weeks ago, but we were still waiting for 50 minutes. During the meal, the waiter (Brent?) forgot our appetizer, and my husband's steak arrived raw. Additionally, we were charged $214, which included an 'automatic' 20% tip for a party of two. I am very unhappy with this experience and will not be returning.
Judge medians: Tone match 9 · Faithfulness 6 · Naturalness 7
Scoring detail
{
"cleaned_words": 80,
"similarity_to_source": 0.478,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.379,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Mellum2.1 12B-A2.5B
✗ fail · score 59 (scripted checks 72, judges 4.7/10 from 3) · 1.21 s
We booked a table for two at 7:30 pm three weeks ago under the name "Okonkwo." We still waited about fifty minutes. The waiter, whose name is Brent, forgot our appetizer, and my husband's steak was served raw. Additionally, we were charged $214, which includes an automatic 20% tip for a party of two. We will not return. Please give us zero stars.
Judge medians: Tone match 6 · Faithfulness 4 · Naturalness 4
Scoring detail
{
"cleaned_words": 63,
"similarity_to_source": 0.479,
"facts_missing": [
"anniversary",
"waited 50 minutes"
],
"banned_hits": [
"zero stars"
],
"new_numbers": [],
"length_ratio": 1.086,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 0.8182,
"banned": 0,
"numbers": 1,
"changed": 1,
"length": 1
}
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 72 (scripted checks 80, judges 6.3/10 from 3) · 0.87 s
We had a disappointing anniversary dinner. We booked a table for 2 at 7:30 under 'Okonkwo' three weeks ago and still waited 50 minutes. The waiter (Brent?) forgot our appetizer, my husband's steak came out raw, and then you charged us $214 including an 'automatic' 20% tip for a party of TWO?!? We will not be returning.
Judge medians: Tone match 4 · Faithfulness 9 · Naturalness 6
Scoring detail
{
"cleaned_words": 57,
"similarity_to_source": 0.817,
"facts_missing": [],
"banned_hits": [
"?!"
],
"new_numbers": [],
"length_ratio": 0.983,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 0,
"numbers": 1,
"changed": 1,
"length": 1
}
}MiniCPM5 2B
✗ fail · score 75 (scripted checks 80, judges 7.0/10 from 3) · 1.28 s
The anniversary dinner we booked for two at 7:30 under 'Okonkwo' three weeks ago proved disappointing. We waited 50 minutes, the waiter (Brent?) forgot our appetizer, my husband's steak came out raw, and we were charged $214, which included an 'automatic' 20% tip for a party of TWO. We will not return, and we would rate zero stars.
Judge medians: Tone match 5 · Faithfulness 10 · Naturalness 6
Scoring detail
{
"cleaned_words": 58,
"similarity_to_source": 0.638,
"facts_missing": [],
"banned_hits": [
"zero stars"
],
"new_numbers": [],
"length_ratio": 1,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 0,
"numbers": 1,
"changed": 1,
"length": 1
}
}K2-Horizon 0.9B
✗ fail · score 48 (scripted checks 35, judges 6.0/10 from 3) · 0.55 s
WORST anniversary dinner EVER. We booked a table for 2 at 7:30 under 'Okonkwo' three weeks ago and still waited 50 minutes. The waiter (Brent?) forgot our appetizer, my husband's steak came out raw, and then you charged us $214 including an 'automatic' 20% tip for a party of TWO?!? Never coming back. Zero stars if I could.
Judge medians: Tone match 0 · Faithfulness 10 · Naturalness 8
Scoring detail
{
"cleaned_words": 58,
"similarity_to_source": 1,
"facts_missing": [],
"banned_hits": [
"worst",
"never coming back",
"zero stars",
"?!",
"ALLCAPS:EVER,WORST"
],
"new_numbers": [],
"length_ratio": 1,
"length_bounds": [
0.5,
3
],
"gate": 0.5,
"checks": {
"facts": 1,
"banned": 0,
"numbers": 1,
"changed": 0,
"length": 1
}
}