Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread
rewrite-003
easy proofreadpost
The prompt
user
Proofread the text below. Fix spelling, grammar, and punctuation mistakes only. Do not rephrase sentences that are already correct, and keep names, numbers, wording, and layout otherwise the same. Reply with only the corrected text, with no preamble and no list of changes. """ Our annual bake sale is this Saterday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options to. Stop by and say hi. Every dollar makes a diffrence! """
What we expected
{
"type": "proofread",
"source": "Our annual bake sale is this Saterday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options to. Stop by and say hi. Every dollar makes a diffrence!",
"errors": [
{
"wrong": "Saterday",
"fixed": [
"Saturday"
]
},
{
"wrong": "options to",
"fixed": [
"options too",
"options, too"
]
},
{
"wrong": "diffrence",
"fixed": [
"difference"
]
}
],
"preserve": [
"Willow Creek Elementary",
"9",
"5th"
],
"must_keep": [],
"length_ratio": [
0.8,
1.25
],
"reference": "Our annual bake sale is this Saturday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options too. Stop by and say hi. Every dollar makes a difference!"
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 · 2.23 s
Our annual bake sale is this Saturday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options too. Stop by and say hi. Every dollar makes a difference!
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.935,
"errors": [
{
"wrong": "Saterday",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "options to",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "diffrence",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 1,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 1,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Ornith-1.5 9B
✓ pass · score 100 · 1.86 s
Our annual bake sale is this Saturday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options too. Stop by and say hi. Every dollar makes a difference!
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.935,
"errors": [
{
"wrong": "Saterday",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "options to",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "diffrence",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 1,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 1,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Granite 4.2 8B
✓ pass · score 100 · 1.94 s
Our annual bake sale is this Saturday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options too. Stop by and say hi. Every dollar makes a difference!
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.935,
"errors": [
{
"wrong": "Saterday",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "options to",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "diffrence",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 1,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 1,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Spark-X2.5 4B
✗ fail · score 80 · 1.25 s
Our annual bake sale is this Saturday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options to. Stop by and say hi. Every dollar makes a difference!
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.957,
"errors": [
{
"wrong": "Saterday",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "options to",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "diffrence",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.978,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.6667,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Mellum2.1 12B-A2.5B
✗ fail · score 80 · 0.89 s
Our annual bake sale is this Saturday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options to. Stop by and say hi. Every dollar makes a difference!
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.957,
"errors": [
{
"wrong": "Saterday",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "options to",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "diffrence",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.978,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.6667,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 80 · 0.62 s
Our annual bake sale is this Saturday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options to. Stop by and say hi. Every dollar makes a difference!
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.957,
"errors": [
{
"wrong": "Saterday",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "options to",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "diffrence",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.978,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.6667,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}MiniCPM5 2B
✗ fail · score 85 · 0.94 s
Our annual bake sale is this Saturday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options. Stop by and say hi. Every dollar makes a difference!
Scoring detail
{
"cleaned_words": 45,
"similarity_to_source": 0.923,
"errors": [
{
"wrong": "Saterday",
"wrong_gone": true,
"fixed_present": true,
"score": 1
},
{
"wrong": "options to",
"wrong_gone": true,
"fixed_present": false,
"score": 0.25
},
{
"wrong": "diffrence",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.967,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 0.978,
"length_bounds": [
0.8,
1.25
],
"gate": 1,
"checks": {
"errors": 0.75,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}K2-Horizon 0.9B
✗ fail · score 40 · 0.36 s
Our annual bake sale is this Saterday from 9 to 1 at Willow Creek Elementary! All proceeds go to the 5th grade field trip. We'll have cookies, brownies, lemon bars and a few gluten-free options to. Stop by and say hi. Every dollar makes a difference!
Scoring detail
{
"cleaned_words": 46,
"similarity_to_source": 0.978,
"errors": [
{
"wrong": "Saterday",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "options to",
"wrong_gone": false,
"fixed_present": false,
"score": 0
},
{
"wrong": "diffrence",
"wrong_gone": true,
"fixed_present": true,
"score": 1
}
],
"fidelity": {
"similarity_to_reference": 0.957,
"unknown_words": [],
"reworded": false,
"unrequested_edits": []
},
"preserve_missing": [],
"length_ratio": 1,
"length_bounds": [
0.8,
1.25
],
"gate": 0.6667,
"checks": {
"errors": 0.3333,
"fidelity": 1,
"preserve": 1,
"length": 1
}
}