Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread

rewrite-015

hard proofreademail

The prompt

user

Proofread the text below. Fix spelling, grammar, and punctuation mistakes only. Do not rephrase sentences that are already correct, and keep names, numbers, wording, and layout otherwise the same. Reply with only the corrected text, with no preamble and no list of changes.

"""
Dear Dr. Lindqvist,

Thank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants which completed all six sessions reported less symptoms at follow-up. We have tried to address every comment from the previous reviewers, who's feedback was very helpful.

The deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to recieve the materials in a different format.

Kind regards,
Mireille Fontaine
Department of Nutrition, Halvorsen University
"""

What we expected

{
  "type": "proofread",
  "source": "Dear Dr. Lindqvist,\n\nThank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants which completed all six sessions reported less symptoms at follow-up. We have tried to address every comment from the previous reviewers, who's feedback was very helpful.\n\nThe deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to recieve the materials in a different format.\n\nKind regards,\nMireille Fontaine\nDepartment of Nutrition, Halvorsen University",
  "errors": [
    {
      "wrong": "participants which completed",
      "fixed": [
        "participants who completed",
        "participants that completed"
      ]
    },
    {
      "wrong": "reported less symptoms",
      "fixed": [
        "reported fewer symptoms"
      ]
    },
    {
      "wrong": "reviewers, who's feedback",
      "fixed": [
        "reviewers, whose feedback"
      ]
    },
    {
      "wrong": "recieve",
      "fixed": [
        "receive"
      ]
    }
  ],
  "preserve": [
    "Lindqvist",
    "Table 2",
    "six sessions",
    "April 30",
    "April 22",
    "Mireille Fontaine",
    "Halvorsen University"
  ],
  "must_keep": [
    "data from our pilot study are summarized"
  ],
  "length_ratio": [
    0.8,
    1.25
  ],
  "reference": "Dear Dr. Lindqvist,\n\nThank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants who completed all six sessions reported fewer symptoms at follow-up. We have tried to address every comment from the previous reviewers, whose feedback was very helpful.\n\nThe deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to receive the materials in a different format.\n\nKind regards,\nMireille Fontaine\nDepartment of Nutrition, Halvorsen University"
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 4.50 s

Dear Dr. Lindqvist,

Thank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants who completed all six sessions reported fewer symptoms at follow-up. We have tried to address every comment from the previous reviewers, whose feedback was very helpful.

The deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to receive the materials in a different format.

Kind regards,
Mireille Fontaine
Department of Nutrition, Halvorsen University

Scoring detail

{
  "cleaned_words": 107,
  "similarity_to_source": 0.963,
  "errors": [
    {
      "wrong": "participants which completed",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "reported less symptoms",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "reviewers, who's feedback",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "recieve",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Ornith-1.5 9B

✓ pass · score 100 · 3.89 s

Dear Dr. Lindqvist,

Thank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants who completed all six sessions reported fewer symptoms at follow-up. We have tried to address every comment from the previous reviewers, whose feedback was very helpful.

The deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to receive the materials in a different format.

Kind regards,
Mireille Fontaine
Department of Nutrition, Halvorsen University

Scoring detail

{
  "cleaned_words": 107,
  "similarity_to_source": 0.963,
  "errors": [
    {
      "wrong": "participants which completed",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "reported less symptoms",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "reviewers, who's feedback",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "recieve",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Granite 4.2 8B

✗ fail · score 86 · 3.97 s

Dear Dr. Lindqvist,

Thank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget, and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants who completed all six sessions reported less symptoms at follow-up. We have tried to address every comment from the previous reviewers, whose feedback was very helpful.

The deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to receive the materials in a different format.

Kind regards,
Mireille Fontaine
Department of Nutrition, Halvorsen University

Scoring detail

{
  "cleaned_words": 107,
  "similarity_to_source": 0.963,
  "errors": [
    {
      "wrong": "participants which completed",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "reported less symptoms",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "reviewers, who's feedback",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "recieve",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.981,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.75,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Spark-X2.5 4B

✗ fail · score 86 · 2.78 s

Dear Dr. Lindqvist,

Thank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget, and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants who completed all six sessions reported less symptoms at follow-up. We have tried to address every comment from the previous reviewers, whose feedback was very helpful.

The deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to receive the materials in a different format.

Kind regards,
Mireille Fontaine
Department of Nutrition, Halvorsen University

Scoring detail

{
  "cleaned_words": 107,
  "similarity_to_source": 0.963,
  "errors": [
    {
      "wrong": "participants which completed",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "reported less symptoms",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "reviewers, who's feedback",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "recieve",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.981,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.75,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Mellum2.1 12B-A2.5B

✗ fail · score 73 · 1.83 s

Dear Dr. Lindqvist,

Thank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants which completed all six sessions reported less symptoms at follow-up. We have tried to address every comment from the previous reviewers, whose feedback was very helpful.

The deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to receive the materials in a different format.

Kind regards,
Mireille Fontaine
Department of Nutrition, Halvorsen University

Scoring detail

{
  "cleaned_words": 107,
  "similarity_to_source": 0.981,
  "errors": [
    {
      "wrong": "participants which completed",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "reported less symptoms",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "reviewers, who's feedback",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "recieve",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.981,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.5,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Ling-3.0-tiny (7.9B-A1.3B)

✗ fail · score 86 · 1.38 s

Dear Dr. Lindqvist,

Thank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants who completed all six sessions reported less symptoms at follow-up. We have tried to address every comment from the previous reviewers, whose feedback was very helpful.

The deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to receive the materials in a different format.

Kind regards,
Mireille Fontaine
Department of Nutrition, Halvorsen University

Scoring detail

{
  "cleaned_words": 107,
  "similarity_to_source": 0.972,
  "errors": [
    {
      "wrong": "participants which completed",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "reported less symptoms",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "reviewers, who's feedback",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "recieve",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.991,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.75,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

MiniCPM5 2B

✗ fail · score 29 · 2.13 s

Dear Dr. Lindqvist,

Thank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants which completed all six sessions reported less symptoms at follow-up. We have tried to address every comment from the previous reviewers, whose feedback was very helpful.

The deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to recieve the materials in a different format.

Kind regards,
Mireille Fontaine
Department of Nutrition, Halvorsen University

Scoring detail

{
  "cleaned_words": 107,
  "similarity_to_source": 0.991,
  "errors": [
    {
      "wrong": "participants which completed",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "reported less symptoms",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "reviewers, who's feedback",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "recieve",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.972,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 0.5,
  "checks": {
    "errors": 0.25,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

K2-Horizon 0.9B

✗ fail · score 0 · 0.75 s

Dear Dr. Lindqvist,

Thank you for agreeing to review our grant application. The attached draft has four sections: background, methods, budget and timeline. The data from our pilot study are summarized in Table 2; they suggest that participants which completed all six sessions reported less symptoms at follow-up. We have tried to address every comment from the previous reviewers, who's feedback was very helpful.

The deadline for submission is April 30, so we would be grateful for your comments by April 22. Please let me know if you would prefer to recieve the materials in a different format.

Kind regards,
Mireille Fontaine
Department of Nutrition, Halvorsen University

Scoring detail

{
  "cleaned_words": 107,
  "similarity_to_source": 1,
  "errors": [
    {
      "wrong": "participants which completed",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "reported less symptoms",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "reviewers, who's feedback",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "recieve",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.963,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 0,
  "checks": {
    "errors": 0,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

← rewrite-014All Rewrite and proofread itemsrewrite-016 →