Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread
rewrite-018
easy toneangry-to-polite
The prompt
user
Rewrite the message below so it is calm and polite, while staying clear about the problem and what the writer wants. Keep every fact (names, dates, times, amounts, reference numbers) as given, and do not add new facts or numbers. Reply with only the rewritten message, with no preamble or explanation. """ Your dog has been barking NONSTOP every single night this week until like 1 AM. I have work at 7 and I'm SICK of it. Do something about it or I'm calling the landlord. - Unit 12 """
What we expected
{
"type": "tone",
"style": "angry-to-polite",
"source": "Your dog has been barking NONSTOP every single night this week until like 1 AM. I have work at 7 and I'm SICK of it. Do something about it or I'm calling the landlord. - Unit 12",
"facts": [
{
"name": "dog barking",
"any": [
"bark",
"barking",
"barks"
]
},
{
"name": "nightly",
"any": [
"night",
"nights",
"nightly",
"evening",
"evenings"
]
},
{
"name": "this week",
"any": [
"this week",
"past week",
"last week"
]
},
{
"name": "1 AM",
"any": [
"1 am",
"1 a.m",
"1:00",
"1am"
]
},
{
"name": "work at 7",
"any": [
"7"
]
},
{
"name": "Unit 12",
"any": [
"unit 12"
]
}
],
"banned": [
"sick of",
"do something about it",
"or i'm calling"
],
"banned_re": [],
"no_caps": true,
"caps_ok": [],
"extra_numbers_ok": [],
"length_ratio": [
0.5,
3
],
"reference": "Hi neighbor,\n\nI wanted to reach out about your dog. It has been barking every night this week, sometimes until around 1 AM. I have to be up for work at 7, so it has been hard to get enough sleep. Would you be able to help keep things quieter in the evenings? I'd really appreciate it.\n\nThank you,\nUnit 12"
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 87 (scripted checks 100, judges 7.3/10 from 3) · 3.07 s
Hello, I wanted to bring to your attention that the dog in Unit 12 has been barking continuously every night this week, until around 1 AM. I have work at 7, and the noise has been difficult for me to manage. Could you please look into this and let me know what steps you can take? If it continues, I may need to contact the landlord. Thank you for your help. - Unit 12
Judge medians: Tone match 10 · Faithfulness 4 · Naturalness 8
Scoring detail
{
"cleaned_words": 73,
"similarity_to_source": 0.396,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 2.028,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Ornith-1.5 9B
✓ pass · score 97 (scripted checks 100, judges 9.3/10 from 3) · 2.65 s
Dear Neighbor, I'm writing to let you know that your dog has been barking continuously every night this week, until around 1 AM. Since I have work starting at 7 AM, this is quite disruptive for me. I would really appreciate it if you could look into this and see what you can do to help. If it continues, I may have to contact the landlord. Thank you for your understanding. Best regards, Unit 12
Judge medians: Tone match 10 · Faithfulness 8 · Naturalness 10
Scoring detail
{
"cleaned_words": 75,
"similarity_to_source": 0.411,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 2.083,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Granite 4.2 8B
✓ pass · score 85 (scripted checks 100, judges 7.0/10 from 3) · 2.88 s
Hello, I hope you are well. I am writing to let you know that your dog has been barking continuously every night this week, generally until around 1 AM. This has been affecting my sleep and making it difficult for me to get adequate rest before my work at 7 AM. Could you please help address the barking issue at your earliest convenience? I would appreciate any assistance you can provide. Thank you for your understanding. Unit 12
Judge medians: Tone match 8 · Faithfulness 5 · Naturalness 8
Scoring detail
{
"cleaned_words": 78,
"similarity_to_source": 0.296,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 2.167,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Spark-X2.5 4B
✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 3) · 1.40 s
I am writing to you regarding the barking your dog has been doing continuously every night this week until approximately 1 AM. Since I have work at 7, I am very frustrated by this situation. Could you please take steps to address this issue? If it is not resolved, I will need to contact the landlord. - Unit 12
Judge medians: Tone match 9 · Faithfulness 10 · Naturalness 8
Scoring detail
{
"cleaned_words": 58,
"similarity_to_source": 0.438,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.611,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Mellum2.1 12B-A2.5B
✓ pass · score 87 (scripted checks 100, judges 7.3/10 from 3) · 0.77 s
Your dog has been barking nonstop every night this week until about 1 AM. I have work at 7 AM and am very tired of it. Please take action about this issue; otherwise, I will call the landlord. — Unit 12
Judge medians: Tone match 6 · Faithfulness 9 · Naturalness 7
Scoring detail
{
"cleaned_words": 40,
"similarity_to_source": 0.692,
"facts_missing": [],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.111,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 1,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 85 (scripted checks 93, judges 7.7/10 from 3) · 0.56 s
I am writing about the noise from your dog. It has been barking nonstop every night this week until around 1 AM. I have work at 7 and I am very tired from the constant noise. Please address this issue, or I will contact the landlord.
Judge medians: Tone match 8 · Faithfulness 7 · Naturalness 8
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.506,
"facts_missing": [
"Unit 12"
],
"banned_hits": [],
"new_numbers": [],
"length_ratio": 1.278,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 0.8333,
"banned": 1,
"numbers": 1,
"changed": 1,
"length": 1
}
}MiniCPM5 2B
✗ fail · score 49 (scripted checks 35, judges 6.3/10 from 3) · 0.78 s
Your dog has been barking nonstop every night this week until around 1 AM. I have work at 7 and I'm sick of it. Do something about it or I'm calling the landlord. - Unit 12
Judge medians: Tone match 1 · Faithfulness 10 · Naturalness 8
Scoring detail
{
"cleaned_words": 35,
"similarity_to_source": 0.959,
"facts_missing": [],
"banned_hits": [
"sick of",
"do something about it",
"or i'm calling"
],
"new_numbers": [],
"length_ratio": 0.972,
"length_bounds": [
0.5,
3
],
"gate": 0.5,
"checks": {
"facts": 1,
"banned": 0,
"numbers": 1,
"changed": 0,
"length": 1
}
}K2-Horizon 0.9B
✗ fail · score 63 (scripted checks 73, judges 5.3/10 from 3) · 0.27 s
Your dog has been barking nonstop every night this week until around 1 AM. I have work at 7 and I'm sick of it. Please do something about it or I may call the landlord.
Judge medians: Tone match 3 · Faithfulness 7 · Naturalness 6
Scoring detail
{
"cleaned_words": 35,
"similarity_to_source": 0.833,
"facts_missing": [
"Unit 12"
],
"banned_hits": [
"sick of",
"do something about it"
],
"new_numbers": [],
"length_ratio": 0.972,
"length_bounds": [
0.5,
3
],
"gate": 1,
"checks": {
"facts": 0.8333,
"banned": 0,
"numbers": 1,
"changed": 1,
"length": 1
}
}