Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread
rewrite-016
hard proofreadspeech
The prompt
user
Proofread the text below. Fix spelling, grammar, and punctuation mistakes only. Do not rephrase sentences that are already correct, and keep names, numbers, wording, and layout otherwise the same. Reply with only the corrected text, with no preamble and no list of changes. """ Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone who's laugh was louder then his. I've never seen him happier. Ingrid, you've had a huge affect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and you're arguments always be short. Please raise you're glasses to Rory and Ingrid! """
What we expected
{
"type": "proofread",
"source": "Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone who's laugh was louder then his. I've never seen him happier. Ingrid, you've had a huge affect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and you're arguments always be short. Please raise you're glasses to Rory and Ingrid!",
"errors": [
{
"wrong": "who's laugh",
"fixed": [
"whose laugh"
]
},
{
"wrong": "louder then",
"fixed": [
"louder than"
]
},
{
"wrong": "huge affect",
"fixed": [
"huge effect"
]
},
{
"wrong": "you're arguments",
"fixed": [
"your arguments"
]
},
{
"wrong": "raise you're glasses",
"fixed": [
"raise your glasses"
]
}
],
"preserve": [
"Callum",
"Rory",
"Ingrid",
"four years",
[
"2 a.m",
"2 am",
"2am",
"2:00 a.m",
"2:00 am"
]
],
"must_keep": [],
"length_ratio": [
0.8,
1.25
],
"reference": "Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone whose laugh was louder than his. I've never seen him happier. Ingrid, you've had a huge effect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and your arguments always be short. Please raise your glasses to Rory and Ingrid!"
}What each model answered
MiMo-V2.6 Distill 9B
✗ fail · score 76 · 4.76 s
Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone who's laugh was louder than his. I've never seen him happier. Ingrid, you've had a huge affect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and your arguments always be short. Please raise your glasses to Rory and Ingrid!
Scoring detail
{
"cleaned_words": 114,
"similarity_to_source": 0.974,
"errors": [
{
"wrong": "who's laugh",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "louder then",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "huge affect",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "you're arguments",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "raise you're glasses",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.982,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.6,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Ornith-1.5 9B
✗ fail · score 88 · 4.00 s
Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone whose laugh was louder than his. I've never seen him happier. Ingrid, you've had a huge affect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and your arguments always be short. Please raise your glasses to Rory and Ingrid!
Scoring detail
{
"cleaned_words": 114,
"similarity_to_source": 0.965,
"errors": [
{
"wrong": "who's laugh",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "louder then",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "huge affect",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "you're arguments",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "raise you're glasses",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.991,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.8,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Granite 4.2 8B
✓ pass · score 100 · 4.02 s
Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone whose laugh was louder than his. I've never seen him happier. Ingrid, you've had a huge effect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and your arguments always be short. Please raise your glasses to Rory and Ingrid!
Scoring detail
{
"cleaned_words": 114,
"similarity_to_source": 0.956,
"errors": [
{
"wrong": "who's laugh",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "louder then",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "huge affect",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "you're arguments",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "raise you're glasses",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 1,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 1,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Spark-X2.5 4B
✗ fail · score 88 · 2.84 s
Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone whose laugh was louder than his. I've never seen him happier. Ingrid, you've had a huge affect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and your arguments always be short. Please raise your glasses to Rory and Ingrid!
Scoring detail
{
"cleaned_words": 114,
"similarity_to_source": 0.965,
"errors": [
{
"wrong": "who's laugh",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "louder then",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "huge affect",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "you're arguments",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "raise you're glasses",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.991,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.8,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Mellum2.1 12B-A2.5B
✗ fail · score 88 · 1.94 s
Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone whose laugh was louder than his. I've never seen him happier. Ingrid, you've had a huge affect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and your arguments always be short. Please raise your glasses to Rory and Ingrid!
Scoring detail
{
"cleaned_words": 114,
"similarity_to_source": 0.965,
"errors": [
{
"wrong": "who's laugh",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "louder then",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "huge affect",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "you're arguments",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "raise you're glasses",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.991,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.8,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 51 · 1.40 s
Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone who's laugh was louder then his. I've never seen him happier. Ingrid, you've had a huge affect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and your arguments always be short. Please raise your glasses to Rory and Ingrid!
Scoring detail
{
"cleaned_words": 114,
"similarity_to_source": 0.982,
"errors": [
{
"wrong": "who's laugh",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "louder then",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "huge affect",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "you're arguments",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "raise you're glasses",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.974,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 0.8,
"checks": {
"errors": 0.4,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}MiniCPM5 2B
✗ fail · score 0 · 2.28 s
Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone who's laugh was louder then his. I've never seen him happier. Ingrid, you've had a huge affect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and you're arguments always be short. Please raise you're glasses to Rory and Ingrid!
Scoring detail
{
"cleaned_words": 114,
"similarity_to_source": 1,
"errors": [
{
"wrong": "who's laugh",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "louder then",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "huge affect",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "you're arguments",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "raise you're glasses",
"wrong_gone": false,
"fixed_present": false,
"score": 0
}
],
"fidelity": {
"similarity_to_reference": 0.956,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 0,
"checks": {
"errors": 0,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}K2-Horizon 0.9B
✗ fail · score 20 · 0.83 s
Good evening, everyone. For those who don't know me, I'm Callum, Rory's older brother. Growing up, Rory and I shared a bedroom, a bicycle, and, unfortunately, a lot of chores. He was always the one who could talk his way out of washing dishes. When he met Ingrid four years ago, he called me at 2 a.m. to say he'd finally met someone who's laugh was louder than his. I've never seen him happier. Ingrid, you've had a huge affect on him: he now makes his bed, he remembers birthdays and he even cooks. May your house always be full and you're arguments always be short. Please raise you're glasses to Rory and Ingrid!
Scoring detail
{
"cleaned_words": 114,
"similarity_to_source": 0.982,
"errors": [
{
"wrong": "who's laugh",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "louder then",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "huge affect",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "you're arguments",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "raise you're glasses",
"wrong_gone": false,
"fixed_present": false,
"score": 0
}
],
"fidelity": {
"similarity_to_reference": 0.956,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 0.4,
"checks": {
"errors": 0.2,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}