Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread
rewrite-020
easy toneformal-to-casual
The prompt
user
Rewrite the message below in a friendly, casual tone, like a message to friends or coworkers. Keep every fact (names, dates, times, amounts, reference numbers) as given, and do not add new facts or numbers. Reply with only the rewritten message, with no preamble or explanation. """ Dear Colleagues, You are cordially invited to attend a farewell gathering in honor of Ms. Beatrice Lam, who will be retiring after 22 years of service. The gathering will take place on Thursday, June 5, at 4:00 p.m. in the third-floor break room. Light refreshments will be served. Your attendance would be greatly appreciated. """
What we expected
{
"type": "tone",
"style": "formal-to-casual",
"source": "Dear Colleagues,\n\nYou are cordially invited to attend a farewell gathering in honor of Ms. Beatrice Lam, who will be retiring after 22 years of service. The gathering will take place on Thursday, June 5, at 4:00 p.m. in the third-floor break room. Light refreshments will be served. Your attendance would be greatly appreciated.",
"facts": [
{
"name": "Beatrice",
"any": [
"beatrice"
]
},
{
"name": "retiring",
"any": [
"retire",
"retiring",
"retirement"
]
},
{
"name": "22 years",
"any": [
"22 years"
]
},
{
"name": "Thursday",
"any": [
"thursday"
]
},
{
"name": "June 5",
"any": [
"june 5"
]
},
{
"name": "4 pm",
"any": [
"4:00",
"4 pm",
"4 p.m",
"4pm"
]
},
{
"name": "third floor",
"any": [
"third floor",
"third-floor",
"3rd floor",
"3rd-floor"
]
},
{
"name": "break room",
"any": [
"break room",
"breakroom"
]
}
],
"banned": [
"cordially",
"dear colleagues",
"greatly appreciated",
"your attendance",
"will take place"
],
"banned_re": [],
"no_caps": false,
"caps_ok": [],
"extra_numbers_ok": [
"3"
],
"length_ratio": [
0.4,
2
],
"reference": "Hey all! Beatrice is retiring after 22 years, and we're throwing her a send-off on Thursday, June 5 at 4:00 p.m. in the third-floor break room. There'll be snacks and drinks. Come say goodbye and celebrate with us!"
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 3) · 2.52 s
Hey everyone, You're warmly invited to a farewell gathering for Ms. Beatrice Lam, who's retiring after 22 years of service. It'll be on Thursday, June 5, at 4:00 p.m. in the third-floor break room, with light refreshments served. Your presence would mean a lot to us!
Judge medians: Tone match 8 · Faithfulness 10 · Naturalness 9
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.6,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 0.852,
"length_bounds": [
0.4,
2
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Ornith-1.5 9B
✓ pass · score 98 (scripted checks 100, judges 9.7/10 from 3) · 2.07 s
Hi everyone, We're hosting a farewell gathering for Beatrice Lam, who's retiring after 22 years of service. The event will be on Thursday, June 5, at 4:00 p.m. in the third-floor break room, and light refreshments will be served. We'd really appreciate it if you could come and say goodbye.
Judge medians: Tone match 9 · Faithfulness 10 · Naturalness 10
Scoring detail
{
"cleaned_words": 50,
"similarity_to_source": 0.558,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 0.926,
"length_bounds": [
0.4,
2
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Granite 4.2 8B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 2.33 s
Hey team, Just a quick invite: we’re having a little farewell gathering for Ms. Beatrice Lam, who’s retiring after 22 years of service. It’ll be on Thursday, June 5 at 4:00 p.m. in the third‑floor break room, with some light refreshments. Hope you can make it!
Judge medians: Tone match 10 · Faithfulness 10 · Naturalness 10
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.46,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 0.852,
"length_bounds": [
0.4,
2
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Spark-X2.5 4B
✗ fail · score 83 (scripted checks 80, judges 8.7/10 from 3) · 1.51 s
Hey everyone! You're all invited to a farewell gathering to celebrate Ms. Beatrice Lam, who is retiring after 22 years of service. It's happening on Thursday, June 5, at 4:00 p.m. in the third-floor break room, and light refreshments will be served. We really appreciate your attendance!
Judge medians: Tone match 8 · Faithfulness 10 · Naturalness 8
Scoring detail
{
"cleaned_words": 47,
"similarity_to_source": 0.634,
"facts_missing": [],
"banned_hits": [
"your attendance"
],
"new_numbers": [],
"length_ratio": 0.87,
"length_bounds": [
0.4,
2
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 0,
"numbers": 1,
"changed": 1,
"length": 1
}
}Mellum2.1 12B-A2.5B
✓ pass · score 93 (scripted checks 100, judges 8.7/10 from 3) · 1.00 s
Hey team, You’re all invited to a farewell party for Beatrice Lam, who’s been retiring after 22 years of service. It’s on Thursday, June 5 at 4:00 p.m. in the third‑floor break room. They’ll have light refreshments. Hope you can make it!
Judge medians: Tone match 10 · Faithfulness 9 · Naturalness 7
Scoring detail
{
"cleaned_words": 42,
"similarity_to_source": 0.5,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 0.778,
"length_bounds": [
0.4,
2
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 97 (scripted checks 94, judges 10.0/10 from 3) · 0.76 s
Hey everyone, We're having a farewell party to say goodbye to Ms. Beatrice Lam, who's retiring after 22 amazing years. It's on Thursday, June 5, at 4:00 p.m. in the third-floor break room. We'll have light snacks and drinks. Please come by — it would mean so much to us!
Judge medians: Tone match 10 · Faithfulness 10 · Naturalness 10
Scoring detail
{
"cleaned_words": 49,
"similarity_to_source": 0.423,
"facts_missing": [
"22 years"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 0.907,
"length_bounds": [
0.4,
2
],
"gate": 1,
"checks": {
"facts": 0.875,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}MiniCPM5 2B
✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 3) · 1.14 s
Hey there! You're all invited to a little farewell gathering to celebrate Ms. Beatrice Lam, who's retiring after 22 years of service. The party will be on Thursday, June 5, at 4:00 p.m. in the third-floor break room. Light refreshments will be served, and your presence would be really appreciated.
Judge medians: Tone match 8 · Faithfulness 10 · Naturalness 9
Scoring detail
{
"cleaned_words": 50,
"similarity_to_source": 0.692,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 0.926,
"length_bounds": [
0.4,
2
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}K2-Horizon 0.9B
✗ fail · score 89 (scripted checks 94, judges 8.3/10 from 3) · 0.41 s
Hey team, We’re sending you a heads-up about Ms. Beatrice Lam’s farewell party on Thursday, June 5, at 4 p.m. in the third-floor break room. She’s been with us for 22 years, and we’d love for you to join us for the refreshments. Your presence means a lot!
Judge medians: Tone match 9 · Faithfulness 7 · Naturalness 9
Scoring detail
{
"cleaned_words": 48,
"similarity_to_source": 0.314,
"facts_missing": [
"retiring"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 0.889,
"length_bounds": [
0.4,
2
],
"gate": 1,
"checks": {
"facts": 0.875,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}