Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread
rewrite-011
medium proofreadletter
The prompt
user
Proofread the text below. Fix spelling, grammar, and punctuation mistakes only. Do not rephrase sentences that are already correct, and keep names, numbers, wording, and layout otherwise the same. Reply with only the corrected text, with no preamble and no list of changes. """ To whom it may concern, On August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be deliverd to my hotel within 24 hours. It has now been six days. The bag contains medication that I take dayly, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190. I would appreciate if you could locate my bag and reimburse me for these costs. Please contact me at the number below at you're earliest convenience. Sincerely, Harold Vance """
What we expected
{
"type": "proofread",
"source": "To whom it may concern,\n\nOn August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be deliverd to my hotel within 24 hours. It has now been six days. The bag contains medication that I take dayly, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190.\n\nI would appreciate if you could locate my bag and reimburse me for these costs. Please contact me at the number below at you're earliest convenience.\n\nSincerely,\nHarold Vance",
"errors": [
{
"wrong": "deliverd",
"fixed": [
"delivered"
]
},
{
"wrong": "dayly",
"fixed": [
"daily"
]
},
{
"wrong": "appreciate if you",
"fixed": [
"appreciate it if you"
]
},
{
"wrong": "at you're earliest",
"fixed": [
"at your earliest"
]
}
],
"preserve": [
"August 3",
"Denver",
"Portland",
"2218",
"24 hours",
"six days",
"$186.40",
"PDX-55190",
"Harold Vance"
],
"must_keep": [],
"length_ratio": [
0.8,
1.25
],
"reference": "To whom it may concern,\n\nOn August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be delivered to my hotel within 24 hours. It has now been six days. The bag contains medication that I take daily, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190.\n\nI would appreciate it if you could locate my bag and reimburse me for these costs. Please contact me at the number below at your earliest convenience.\n\nSincerely,\nHarold Vance"
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 · 4.99 s
To whom it may concern, On August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be delivered to my hotel within 24 hours. It has now been six days. The bag contains medication that I take daily, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190. I would appreciate it if you could locate my bag and reimburse me for these costs. Please contact me at the number below at your earliest convenience. Sincerely, Harold Vance
Scoring detail
{
"cleaned_words": 108,
"similarity_to_source": 0.967,
"errors": [
{
"wrong": "deliverd",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "dayly",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "appreciate if you",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "at you're earliest",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 1,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1.009,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 1,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Ornith-1.5 9B
✓ pass · score 100 · 3.88 s
To whom it may concern, On August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be delivered to my hotel within 24 hours. It has now been six days. The bag contains medication that I take daily, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190. I would appreciate it if you could locate my bag and reimburse me for these costs. Please contact me at the number below at your earliest convenience. Sincerely, Harold Vance
Scoring detail
{
"cleaned_words": 108,
"similarity_to_source": 0.967,
"errors": [
{
"wrong": "deliverd",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "dayly",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "appreciate if you",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "at you're earliest",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 1,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1.009,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 1,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Granite 4.2 8B
✗ fail · score 85 · 3.90 s
To whom it may concern, On August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be delivered to my hotel within 24 hours. It has now been six days. The bag contains medication that I take daily, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190. I would appreciate if you could locate my bag and reimburse me for these costs. Please contact me at the number below at your earliest convenience. Sincerely, Harold Vance
Scoring detail
{
"cleaned_words": 107,
"similarity_to_source": 0.972,
"errors": [
{
"wrong": "deliverd",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "dayly",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "appreciate if you",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "at you're earliest",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.995,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.75,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Spark-X2.5 4B
✓ pass · score 100 · 3.00 s
To whom it may concern, On August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be delivered to my hotel within 24 hours. It has now been six days. The bag contains medication that I take daily, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190. I would appreciate it if you could locate my bag and reimburse me for these costs. Please contact me at the number below at your earliest convenience. Sincerely, Harold Vance
Scoring detail
{
"cleaned_words": 108,
"similarity_to_source": 0.967,
"errors": [
{
"wrong": "deliverd",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "dayly",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "appreciate if you",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "at you're earliest",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 1,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1.009,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 1,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Mellum2.1 12B-A2.5B
✓ pass · score 100 · 1.93 s
To whom it may concern, On August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be delivered to my hotel within 24 hours. It has now been six days. The bag contains medication that I take daily, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190. I would appreciate it if you could locate my bag and reimburse me for these costs. Please contact me at the number below at your earliest convenience. Sincerely, Harold Vance
Scoring detail
{
"cleaned_words": 108,
"similarity_to_source": 0.967,
"errors": [
{
"wrong": "deliverd",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "dayly",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "appreciate if you",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "at you're earliest",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 1,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1.009,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 1,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 85 · 1.42 s
To whom it may concern, On August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be delivered to my hotel within 24 hours. It has now been six days. The bag contains medication that I take daily, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190. I would appreciate if you could locate my bag and reimburse me for these costs. Please contact me at the number below at your earliest convenience. Sincerely, Harold Vance
Scoring detail
{
"cleaned_words": 107,
"similarity_to_source": 0.972,
"errors": [
{
"wrong": "deliverd",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "dayly",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "appreciate if you",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "at you're earliest",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.995,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.75,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}MiniCPM5 2B
✗ fail · score 85 · 2.10 s
To whom it may concern, On August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be delivered to my hotel within 24 hours. It has now been six days. The bag contains medication that I take daily, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190. I would appreciate if you could locate my bag and reimburse me for these costs. Please contact me at the number below at your earliest convenience. Sincerely, Harold Vance
Scoring detail
{
"cleaned_words": 107,
"similarity_to_source": 0.972,
"errors": [
{
"wrong": "deliverd",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "dayly",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "appreciate if you",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "at you're earliest",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.995,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.75,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}K2-Horizon 0.9B
✗ fail · score 69 · 0.73 s
To whom it may concern, On August 3, I flew from Denver to Portland on flight 2218. My checked bag did not arrive, and I was told it would be delivered to my hotel within 24 hours. It has now been six days. The bag contains medication that I take daily, and I have had to buy replacements at my own expense. I have attached the receipts, which total $186.40, as well as my baggage claim number, PDX-55190. I would appreciate if you could locate my bag and reimburse me for these costs. Please contact me at the number below at you're earliest convenience. Sincerely, Harold Vance
Scoring detail
{
"cleaned_words": 107,
"similarity_to_source": 0.981,
"errors": [
{
"wrong": "deliverd",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "dayly",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "appreciate if you",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "at you're earliest",
"wrong_gone": false,
"fixed_present": false,
"score": 0
}
],
"fidelity": {
"similarity_to_reference": 0.986,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.5,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}