Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread
rewrite-032
hard tonecurt-to-warm
The prompt
user
Rewrite the message below so it sounds warm and friendly while keeping the same information. Keep every fact (names, dates, times, amounts, reference numbers) as given, and do not add new facts or numbers. Reply with only the rewritten message, with no preamble or explanation. """ Tournament Sat. Bus leaves school 6:15 AM sharp. Late = left behind. Bring: uniform (both jerseys), cleats, water, $15 for lunch. Pickup ~5 PM at school. No phones on the bus. Questions to Coach Ramirez only. """
What we expected
{
"type": "tone",
"style": "curt-to-warm",
"source": "Tournament Sat. Bus leaves school 6:15 AM sharp. Late = left behind. Bring: uniform (both jerseys), cleats, water, $15 for lunch. Pickup ~5 PM at school. No phones on the bus. Questions to Coach Ramirez only.",
"facts": [
{
"name": "tournament",
"any": [
"tournament"
]
},
{
"name": "Saturday",
"any": [
"saturday",
"sat"
]
},
{
"name": "6:15",
"any": [
"6:15"
]
},
{
"name": "bus",
"any": [
"bus"
]
},
{
"name": "uniform",
"any": [
"uniform"
]
},
{
"name": "both jerseys",
"any": [
"both jerseys",
"two jerseys",
"2 jerseys",
"both of their jerseys",
"both their jerseys",
"both of your jerseys",
"both your jerseys",
"both of the jerseys",
"both uniform jerseys",
"both team jerseys"
]
},
{
"name": "cleats",
"any": [
"cleats"
]
},
{
"name": "water",
"any": [
"water"
]
},
{
"name": "$15 lunch",
"any": [
"$15"
]
},
{
"name": "5 pm pickup",
"any": [
"5 pm",
"5pm",
"5:00",
"5 p.m"
]
},
{
"name": "pickup",
"any": [
"pickup",
"pick up",
"pick-up"
]
},
{
"name": "phones",
"any": [
"phone",
"phones"
]
},
{
"name": "Coach Ramirez",
"any": [
"ramirez"
]
}
],
"banned": [
"late = left behind",
"late ="
],
"banned_re": [],
"no_caps": false,
"caps_ok": [],
"extra_numbers_ok": [
"2"
],
"length_ratio": [
1.2,
6
],
"reference": "Hi families! Our tournament is this Saturday, and we're so excited. The bus will leave from school at 6:15 AM, so please have your player there a little early; we won't be able to wait for late arrivals. Please pack their uniform (both jerseys), cleats, water, and $15 for lunch. We expect to be back at school for pickup around 5 PM. Phones aren't allowed on the bus, so please have them packed away. If you have any questions, please reach out to Coach Ramirez. Thank you, and go team!"
}What each model answered
MiMo-V2.6 Distill 9B
✗ fail · score 87 (scripted checks 97, judges 7.7/10 from 3) · 3.12 s
Hi everyone! Just a quick reminder: the tournament Saturday bus leaves school at 6:15 AM sharp, and if you're late, you'll be left behind. Please bring your uniform (both jerseys), cleats, water, and $15 for lunch. We'll be picking up around 5 PM at school. Also, no phones on the bus. If you have any questions, please reach out to Coach Ramirez only. Have a great tournament!
Judge medians: Tone match 8 · Faithfulness 8 · Naturalness 7
Scoring detail
{
"cleaned_words": 67,
"similarity_to_source": 0.544,
"facts_missing": [
"pickup"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.914,
"length_bounds": [
1.2,
6
],
"gate": 1,
"checks": {
"facts": 0.9231,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Ornith-1.5 9B
✗ fail · score 84 (scripted checks 93, judges 7.5/10 from 2) · 2.90 s
Hi there! We're excited to have you join us for the tournament. The bus will leave school at 6:15 AM sharp, so please make sure to be ready on time—anyone who's late will be left behind. Please bring your uniform (both jerseys), cleats, and water, along with $15 for lunch. We'll pick you up around 5 PM at the school. Also, please remember that phones are not allowed on the bus. If you have any questions, feel free to reach out to Coach Ramirez. Have a great time!
Judge medians: Tone match 9.5 · Faithfulness 4.5 · Naturalness 8.5
Scoring detail
{
"cleaned_words": 88,
"similarity_to_source": 0.387,
"facts_missing": [
"Saturday",
"pickup"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 2.514,
"length_bounds": [
1.2,
6
],
"gate": 1,
"checks": {
"facts": 0.8462,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Granite 4.2 8B
✗ fail · score 85 (scripted checks 97, judges 7.3/10 from 3) · 2.60 s
Hello! Just a quick reminder about Saturday’s tournament: the bus leaves school at 6:15 AM sharp, so please be on time—if you’re late you’ll be left behind. Please bring your uniform (both jerseys), cleats, water, and $15 for lunch. We’ll pick you up back at school around 5 PM. No phones on the bus. If you have any questions, please reach out to Coach Ramirez only.
Judge medians: Tone match 7 · Faithfulness 7 · Naturalness 8
Scoring detail
{
"cleaned_words": 66,
"similarity_to_source": 0.51,
"facts_missing": [
"pickup"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.886,
"length_bounds": [
1.2,
6
],
"gate": 1,
"checks": {
"facts": 0.9231,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Spark-X2.5 4B
✗ fail · score 87 (scripted checks 97, judges 7.7/10 from 3) · 1.83 s
Here's the important info for the tournament! The bus will leave the school at 6:15 AM sharp on Saturday. Please be on time, as we won't be able to wait for latecomers. Remember to bring your uniform (both jerseys), cleats, water, and $15 for lunch. We'll be picking you up around 5 PM at school. No phones are allowed on the bus, and if you have any questions, please reach out to Coach Ramirez.
Judge medians: Tone match 9 · Faithfulness 5 · Naturalness 9
Scoring detail
{
"cleaned_words": 74,
"similarity_to_source": 0.382,
"facts_missing": [
"pickup"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 2.114,
"length_bounds": [
1.2,
6
],
"gate": 1,
"checks": {
"facts": 0.9231,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Mellum2.1 12B-A2.5B
✗ fail · score 70 (scripted checks 90, judges 5.0/10 from 3) · 1.02 s
The tournament bus leaves school at 6:15 AM sharp—late means you’ll be left behind. Please bring: both jerseys, cleats, a water bottle, and $15 for lunch. We’ll pick you up around 5 PM at school. No phones on the bus. Any questions? Please ask Coach Ramirez.
Judge medians: Tone match 4 · Faithfulness 4 · Naturalness 7
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.537,
"facts_missing": [
"Saturday",
"uniform",
"pickup"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.314,
"length_bounds": [
1.2,
6
],
"gate": 1,
"checks": {
"facts": 0.7692,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 33 (scripted checks 15, judges 5.0/10 from 3) · 0.59 s
Tournament Sat. Bus leaves school 6:15 AM sharp. Late = left behind. Bring: uniform (both jerseys), cleats, water, $15 for lunch. Pickup ~5 PM at school. No phones on the bus. Questions to Coach Ramirez only.
Judge medians: Tone match 0 · Faithfulness 10 · Naturalness 5
Scoring detail
{
"cleaned_words": 35,
"similarity_to_source": 1,
"facts_missing": [],
"banned_hits": [
"late = left behind",
"late ="
],
"new_numbers": [],
"length_ratio": 1,
"length_bounds": [
1.2,
6
],
"gate": 0.25,
"checks": {
"facts": 1,
"banned": 0,
"numbers": 1,
"changed": 0,
"length": 0
}
}MiniCPM5 2B
✗ fail · score 75 (scripted checks 93, judges 5.7/10 from 3) · 1.26 s
The bus leaves school at 6:15 AM sharp, so please make sure to get on before then—late means you'll be left behind. Bring your uniform (both jerseys), cleats, water, and $15 for lunch. Pickup is around 5 PM at school, and please keep phones off the bus. If you have any questions, Coach Ramirez is the only person to ask.
Judge medians: Tone match 4 · Faithfulness 5 · Naturalness 8
Scoring detail
{
"cleaned_words": 60,
"similarity_to_source": 0.479,
"facts_missing": [
"tournament",
"Saturday"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.714,
"length_bounds": [
1.2,
6
],
"gate": 1,
"checks": {
"facts": 0.8462,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}K2-Horizon 0.9B
✗ fail · score 77 (scripted checks 100, judges 5.3/10 from 3) · 0.41 s
Tournament Sat. Bus leaves school at 6:15 AM sharp. Late means you’ll be left behind. Bring your uniform (both jerseys), cleats, water, and $15 for lunch. Pickup is around 5 PM at school. No phones on the bus. Questions are to Coach Ramirez only.
Judge medians: Tone match 1 · Faithfulness 10 · Naturalness 5
Scoring detail
{
"cleaned_words": 44,
"similarity_to_source": 0.825,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.257,
"length_bounds": [
1.2,
6
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}