Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread

rewrite-009

medium proofreademail

The prompt

user

Proofread the text below. Fix spelling, grammar, and punctuation mistakes only. Do not rephrase sentences that are already correct, and keep names, numbers, wording, and layout otherwise the same. Reply with only the corrected text, with no preamble and no list of changes.

"""
Hi Ms. Okafor,

Jaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall to far behind. Also, he mentioned theirs a field trip form due Friday. I think we may of lost ours, so could you send another copy?

Thanks so much,
Andrea Pike
"""

What we expected

{
  "type": "proofread",
  "source": "Hi Ms. Okafor,\n\nJaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall to far behind. Also, he mentioned theirs a field trip form due Friday. I think we may of lost ours, so could you send another copy?\n\nThanks so much,\nAndrea Pike",
  "errors": [
    {
      "wrong": "fall to far",
      "fixed": [
        "fall too far"
      ]
    },
    {
      "wrong": "theirs a field trip",
      "fixed": [
        "there's a field trip",
        "there is a field trip"
      ]
    },
    {
      "wrong": "may of lost",
      "fixed": [
        "may have lost",
        "might have lost"
      ]
    }
  ],
  "preserve": [
    "Okafor",
    "Jaylen",
    "Monday",
    "Tuesday",
    "Friday",
    "Andrea Pike"
  ],
  "must_keep": [],
  "length_ratio": [
    0.8,
    1.25
  ],
  "reference": "Hi Ms. Okafor,\n\nJaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall too far behind. Also, he mentioned there's a field trip form due Friday. I think we may have lost ours, so could you send another copy?\n\nThanks so much,\nAndrea Pike"
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 3.43 s

Hi Ms. Okafor,

Jaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall too far behind. Also, he mentioned there's a field trip form due Friday. I think we may have lost ours, so could you send another copy?

Thanks so much,
Andrea Pike

Scoring detail

{
  "cleaned_words": 76,
  "similarity_to_source": 0.961,
  "errors": [
    {
      "wrong": "fall to far",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "theirs a field trip",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "may of lost",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "length": 1
  }
}

Ornith-1.5 9B

✓ pass · score 100 · 2.80 s

Hi Ms. Okafor,

Jaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall too far behind. Also, he mentioned there's a field trip form due Friday. I think we may have lost ours, so could you send another copy?

Thanks so much,
Andrea Pike

Scoring detail

{
  "cleaned_words": 76,
  "similarity_to_source": 0.961,
  "errors": [
    {
      "wrong": "fall to far",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "theirs a field trip",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "may of lost",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "length": 1
  }
}

Granite 4.2 8B

✓ pass · score 100 · 2.91 s

Hi Ms. Okafor,

Jaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall too far behind. Also, he mentioned there is a field trip form due Friday. I think we may have lost ours, so could you send another copy?

Thanks so much,
Andrea Pike

Scoring detail

{
  "cleaned_words": 77,
  "similarity_to_source": 0.954,
  "errors": [
    {
      "wrong": "fall to far",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "theirs a field trip",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "may of lost",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.98,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "length_ratio": 1.013,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "length": 1
  }
}

Spark-X2.5 4B

✗ fail · score 85 · 2.00 s

Hi Ms. Okafor,

Jaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall too far behind. Also, he mentioned there was a field trip form due Friday. I think we may have lost ours, so could you send another copy?

Thanks so much,
Andrea Pike

Scoring detail

{
  "cleaned_words": 77,
  "similarity_to_source": 0.954,
  "errors": [
    {
      "wrong": "fall to far",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "theirs a field trip",
      "wrong_gone": true,
      "fixed_present": false,
      "score": 0.25
    },
    {
      "wrong": "may of lost",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.98,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "length_ratio": 1.013,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.75,
    "fidelity": 1,
    "preserve": 1,
    "length": 1
  }
}

Mellum2.1 12B-A2.5B

✗ fail · score 85 · 1.31 s

Hi Ms. Okafor,

Jaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall behind. Also, he mentioned there is a field trip form due Friday. I think we may have lost ours, so could you send another copy?

Thanks so much,
Andrea Pike

Scoring detail

{
  "cleaned_words": 75,
  "similarity_to_source": 0.954,
  "errors": [
    {
      "wrong": "fall to far",
      "wrong_gone": true,
      "fixed_present": false,
      "score": 0.25
    },
    {
      "wrong": "theirs a field trip",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "may of lost",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.967,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "length_ratio": 0.987,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.75,
    "fidelity": 1,
    "preserve": 1,
    "length": 1
  }
}

Ling-3.0-tiny (7.9B-A1.3B)

✗ fail · score 85 · 0.99 s

Hi Ms. Okafor,

Jaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall too far behind. Also, he mentioned there was a field trip form due Friday. I think we may have lost ours, so could you send another copy?

Thanks so much,
Andrea Pike

Scoring detail

{
  "cleaned_words": 77,
  "similarity_to_source": 0.954,
  "errors": [
    {
      "wrong": "fall to far",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "theirs a field trip",
      "wrong_gone": true,
      "fixed_present": false,
      "score": 0.25
    },
    {
      "wrong": "may of lost",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.98,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "length_ratio": 1.013,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.75,
    "fidelity": 1,
    "preserve": 1,
    "length": 1
  }
}

MiniCPM5 2B

✗ fail · score 54 · 1.32 s

Hi Ms. Okafor,

Jaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall far behind. Also, he mentioned theirs a field trip form due Friday. I think we may have lost ours, so could you send another copy?

Thanks so much,
Andrea Pike

Scoring detail

{
  "cleaned_words": 75,
  "similarity_to_source": 0.98,
  "errors": [
    {
      "wrong": "fall to far",
      "wrong_gone": true,
      "fixed_present": false,
      "score": 0.25
    },
    {
      "wrong": "theirs a field trip",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "may of lost",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.98,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "length_ratio": 0.987,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 0.8333,
  "checks": {
    "errors": 0.4167,
    "fidelity": 1,
    "preserve": 1,
    "length": 1
  }
}

K2-Horizon 0.9B

✗ fail · score 0 · 0.53 s

Hi Ms. Okafor,

Jaylen was absent on Monday and Tuesday because of a fever. He's feeling much better now and will be back in class tomorrow. Could you please let me know what assignments he missed, especially for science? I don't want him to fall to far behind. Also, he mentioned theirs a field trip form due Friday. I think we may of lost ours, so could you send another copy?

Thanks so much,
Andrea Pike

Scoring detail

{
  "cleaned_words": 76,
  "similarity_to_source": 1,
  "errors": [
    {
      "wrong": "fall to far",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "theirs a field trip",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "may of lost",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.961,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 0,
  "checks": {
    "errors": 0,
    "fidelity": 1,
    "preserve": 1,
    "length": 1
  }
}

← rewrite-008All Rewrite and proofread itemsrewrite-010 →