Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread

rewrite-010

medium proofreadnotes

The prompt

user

Proofread the text below. Fix spelling, grammar, and punctuation mistakes only. Do not rephrase sentences that are already correct, and keep names, numbers, wording, and layout otherwise the same. Reply with only the corrected text, with no preamble and no list of changes.

"""
Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should of tracked drop-off sooner.
- Ravi will look into weather the onboarding email is being sent to spam.
- Design will accomodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.
"""

What we expected

{
  "type": "proofread",
  "source": "Notes - Product sync, Oct 7\n- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.\n- The beta had 312 signups; less than half completed onboarding. We should of tracked drop-off sooner.\n- Ravi will look into weather the onboarding email is being sent to spam.\n- Design will accomodate the new pricing table on the landing page.\n- Fewer than 20 bugs remain open; none are blockers.",
  "errors": [
    {
      "wrong": "should of tracked",
      "fixed": [
        "should have tracked"
      ]
    },
    {
      "wrong": "into weather",
      "fixed": [
        "into whether"
      ]
    },
    {
      "wrong": "accomodate",
      "fixed": [
        "accommodate"
      ]
    }
  ],
  "preserve": [
    "Oct 7",
    "November 12",
    "Oct 28",
    "312",
    "Ravi"
  ],
  "must_keep": [
    "less than half",
    "Fewer than 20"
  ],
  "length_ratio": [
    0.8,
    1.25
  ],
  "reference": "Notes - Product sync, Oct 7\n- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.\n- The beta had 312 signups; less than half completed onboarding. We should have tracked drop-off sooner.\n- Ravi will look into whether the onboarding email is being sent to spam.\n- Design will accommodate the new pricing table on the landing page.\n- Fewer than 20 bugs remain open; none are blockers."
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 3.49 s

Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should have tracked drop-off sooner.
- Ravi will look into whether the onboarding email is being sent to spam.
- Design will accommodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.

Scoring detail

{
  "cleaned_words": 67,
  "similarity_to_source": 0.959,
  "errors": [
    {
      "wrong": "should of tracked",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "into weather",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "accomodate",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Ornith-1.5 9B

✓ pass · score 100 · 2.82 s

Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should have tracked drop-off sooner.
- Ravi will look into whether the onboarding email is being sent to spam.
- Design will accommodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.

Scoring detail

{
  "cleaned_words": 67,
  "similarity_to_source": 0.959,
  "errors": [
    {
      "wrong": "should of tracked",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "into weather",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "accomodate",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

"""
Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should of tracked drop-off sooner.
- Ravi will look into weather the onboarding email is being sent to spam.
- Design will accomodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.
"""

Wait, I need to check carefully. The instruction: "Fix spelling, grammar, and punctuation mistakes only. Do not rephrase sentences that are already correct, and keep names, numbers, wording, and layout otherwise the same."

Let's examine each line:

"Notes - Product sync, Oct 7" seems fine.

"- Launch date stays at November 12. Marketing needs final screenshots by Oct 28." Fine.

"- The beta had 312 signups; less than half completed onboarding. We should of tracked drop-off sooner."

"We should of tracked" is incorrect; it should be "We should have tracked" or "We should've tracked". But "We should of" is a common mistake; the correct phrase is "We should have tracked". However, the instruction says "Do not rephrase sentences that are already correct". But this sentence contains a mistake: "We should of tracked". We need to fix it. So we should change "We should of tracked" to "We should have tracked". But is that considered rephrasing? It's fixing a grammar mistake. Yes, we need to fix it.

But careful: "We should of tracked" -> "We should have tracked". That's a correction.

Next: "- Ravi will look into weather the onboarding email is being sent to spam." "look into weather" is likely a mistake; should be "look into whether". "look into weather" doesn't make sense; it should be "look into whether". So fix to "Ravi will look into whether the onboarding email is being sent to spam."

Next: "- Design will accomodate the new pricing table on the landing page." "accomodate" is misspelled; should be "accommodate". So fix to "Design will accommodate".

Next: "- Fewer than 20 bugs remain open; none are blockers." Fine.

Now check punctuation: The semicolon usage seems fine.

Now we need to output only the corrected text, with no preamble.

But we must ensure we only fix spelling, grammar, punctuation mistakes. Let's list them:

1. "We should of tracked" -> "We should have tracked". (Grammar)

2. "look into weather" -> "look into whether". (Grammar)

3. "accomodate" -> "accommodate". (Spelling)

Also check "Oct 7" and "November 12" are fine.

Now check if any punctuation issues: The semicolon after "signups;" and after "open;" are fine.

Now we need to output the corrected text exactly as the original layout, with corrections.

Original text:

We need to output:

"""
Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should have tracked drop-off sooner.
- Ravi will look into whether the onboarding email is being sent to spam.
- Design will accommodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.
"""

But note: The original had "We should of tracked drop-off sooner." We change "of" to "have". So "We should have tracked drop-off sooner." That seems correct.

But check: "We should have tracked drop-off sooner." Might be ambiguous but it's fixing the mistake.

Now "look into weather" -> "look into whether". Yes.

Now "accomodate" -> "accommodate". Yes.

Now ensure we don't change any other wording. For example, "drop-off" is fine.

Scoring detail

{
  "cleaned_words": 366,
  "similarity_to_source": 0,
  "errors": [
    {
      "wrong": "should of tracked",
      "wrong_gone": false,
      "fixed_present": true,
      "score": 0.25
    },
    {
      "wrong": "into weather",
      "wrong_gone": false,
      "fixed_present": true,
      "score": 0.25
    },
    {
      "wrong": "accomodate",
      "wrong_gone": false,
      "fixed_present": true,
      "score": 0.25
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0,
    "unknown_words": [
      "a",
      "already",
      "and",
      "be",
      "but",
      "careful",
      "carefully",
      "change",
      "check",
      "common",
      "considered",
      "contains",
      "correct",
      "corrected",
      "correction"
    ],
    "reworded": true
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 5.463,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 0.375,
  "checks": {
    "errors": 0.25,
    "fidelity": 0,
    "preserve": 1,
    "must_keep": 1,
    "length": 0
  }
}

Spark-X2.5 4B

✓ pass · score 100 · 2.08 s

Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should have tracked drop-off sooner.
- Ravi will look into whether the onboarding email is being sent to spam.
- Design will accommodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.

Scoring detail

{
  "cleaned_words": 67,
  "similarity_to_source": 0.959,
  "errors": [
    {
      "wrong": "should of tracked",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "into weather",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "accomodate",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Mellum2.1 12B-A2.5B

✓ pass · score 100 · 1.31 s

Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should have tracked drop-off sooner.
- Ravi will look into whether the onboarding email is being sent to spam.
- Design will accommodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.

Scoring detail

{
  "cleaned_words": 67,
  "similarity_to_source": 0.959,
  "errors": [
    {
      "wrong": "should of tracked",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "into weather",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "accomodate",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Ling-3.0-tiny (7.9B-A1.3B)

✓ pass · score 100 · 1.00 s

Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should have tracked drop-off sooner.
- Ravi will look into whether the onboarding email is being sent to spam.
- Design will accommodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.

Scoring detail

{
  "cleaned_words": 67,
  "similarity_to_source": 0.959,
  "errors": [
    {
      "wrong": "should of tracked",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "into weather",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "accomodate",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

MiniCPM5 2B

✓ pass · score 100 · 1.47 s

Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should have tracked drop-off sooner.
- Ravi will look into whether the onboarding email is being sent to spam.
- Design will accommodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.

Scoring detail

{
  "cleaned_words": 67,
  "similarity_to_source": 0.959,
  "errors": [
    {
      "wrong": "should of tracked",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "into weather",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "accomodate",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

K2-Horizon 0.9B

✗ fail · score 0 · 0.53 s

Notes - Product sync, Oct 7
- Launch date stays at November 12. Marketing needs final screenshots by Oct 28.
- The beta had 312 signups; less than half completed onboarding. We should of tracked drop-off sooner.
- Ravi will look into weather the onboarding email is being sent to spam.
- Design will accomodate the new pricing table on the landing page.
- Fewer than 20 bugs remain open; none are blockers.

Scoring detail

{
  "cleaned_words": 67,
  "similarity_to_source": 1,
  "errors": [
    {
      "wrong": "should of tracked",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "into weather",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "accomodate",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.959,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 0,
  "checks": {
    "errors": 0,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

← rewrite-009All Rewrite and proofread itemsrewrite-011 →