Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Rewrite and proofread

rewrite-014

hard proofreademail

The prompt

user

Proofread the text below. Fix spelling, grammar, and punctuation mistakes only. Do not rephrase sentences that are already correct, and keep names, numbers, wording, and layout otherwise the same. Reply with only the corrected text, with no preamble and no list of changes.

"""
Hi Ana,

Thanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:

1. The principle concern from the last meeting was cost, so lets lead with the $412,000 estimate and the phased payment plan.
2. The new lighting plan really compliments the open floor layout. I'd keep the renderings on page 6.
3. We need to be discrete about the subcontractor change until the contract is signed.
4. There are less risks with the steel supplier we chose, but the timeline still depends on there delivery date of March 3.

Could you lay out the final numbers in a single table? I'll review it tonight.

Best,
Tomás
"""

What we expected

{
"type": "proofread",
"source": "Hi Ana,\n\nThanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:\n\n1. The principle concern from the last meeting was cost, so lets lead with the $412,000 estimate and the phased payment plan.\n2. The new lighting plan really compliments the open floor layout. I'd keep the renderings on page 6.\n3. We need to be discrete about the subcontractor change until the contract is signed.\n4. There are less risks with the steel supplier we chose, but the timeline still depends on there delivery date of March 3.\n\nCould you lay out the final numbers in a single table? I'll review it tonight.\n\nBest,\nTomás",
"errors": [
{
"wrong": "principle concern",
"fixed": [
"principal concern"
]
},
{
"wrong": "lets lead",
"fixed": [
"let's lead",
"let us lead"
]
},
{
"wrong": "compliments the open",
"fixed": [
"complements the open"
]
},
{
"wrong": "be discrete",
"fixed": [
"be discreet"
]
},
{
"wrong": "less risks",
"fixed": [
"fewer risks",
"is less risk"
]
},
{
"wrong": "on there delivery",
"fixed": [
"on their delivery"
]
}
],
"preserve": [
"Ana",
"Carver Street",
"Wednesday",
"$412,000",
"page 6",
"March 3",
"Tomás"
],
"must_keep": [
"lay out the final numbers"
],
"length_ratio": [
0.8,
1.25
],
"reference": "Hi Ana,\n\nThanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:\n\n1. The principal concern from the last meeting was cost, so let's lead with the $412,000 estimate and the phased payment plan.\n2. The new lighting plan really complements the open floor layout. I'd keep the renderings on page 6.\n3. We need to be discreet about the subcontractor change until the contract is signed.\n4. There are fewer risks with the steel supplier we chose, but the timeline still depends on their delivery date of March 3.\n\nCould you lay out the final numbers in a single table? I'll review it tonight.\n\nBest,\nTomás"
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 5.30 s

Hi Ana,

Thanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:

1. The principal concern from the last meeting was cost, so let's lead with the $412,000 estimate and the phased payment plan.
2. The new lighting plan really complements the open floor layout. I'd keep the renderings on page 6.
3. We need to be discreet about the subcontractor change until the contract is signed.
4. There are fewer risks with the steel supplier we chose, but the timeline still depends on their delivery date of March 3.

Could you lay out the final numbers in a single table? I'll review it tonight.

Best,
Tomás

Scoring detail

{
  "cleaned_words": 121,
  "similarity_to_source": 0.95,
  "errors": [
    {
      "wrong": "principle concern",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "lets lead",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "compliments the open",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "be discrete",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "less risks",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "on there delivery",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Ornith-1.5 9B

✓ pass · score 100 · 4.45 s

Hi Ana,

Thanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:

1. The principal concern from the last meeting was cost, so let's lead with the $412,000 estimate and the phased payment plan.
2. The new lighting plan really complements the open floor layout. I'd keep the renderings on page 6.
3. We need to be discreet about the subcontractor change until the contract is signed.
4. There are fewer risks with the steel supplier we chose, but the timeline still depends on their delivery date of March 3.

Could you lay out the final numbers in a single table? I'll review it tonight.

Best,
Tomás

Scoring detail

{
  "cleaned_words": 121,
  "similarity_to_source": 0.95,
  "errors": [
    {
      "wrong": "principle concern",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "lets lead",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "compliments the open",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "be discrete",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "less risks",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "on there delivery",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Granite 4.2 8B

✓ pass · score 100 · 4.47 s

Hi Ana,

Thanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:

1. The principal concern from the last meeting was cost, so let's lead with the $412,000 estimate and the phased payment plan.
2. The new lighting plan really complements the open floor layout. I'd keep the renderings on page 6.
3. We need to be discreet about the subcontractor change until the contract is signed.
4. There are fewer risks with the steel supplier we chose, but the timeline still depends on their delivery date of March 3.

Could you lay out the final numbers in a single table? I'll review it tonight.

Best,
Tomás

Scoring detail

{
  "cleaned_words": 121,
  "similarity_to_source": 0.95,
  "errors": [
    {
      "wrong": "principle concern",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "lets lead",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "compliments the open",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "be discrete",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "less risks",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "on there delivery",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 1,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 1,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Spark-X2.5 4B

✗ fail · score 84 · 3.15 s

Hi Ana,

Thanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:

1. The principal concern from the last meeting was cost, so let's lead with the $412,000 estimate and the phased payment plan.
2. The new lighting plan really compliments the open floor layout. I'd keep the renderings on page 6.
3. We need to be discreet about the subcontractor change until the contract is signed.
4. There are fewer risks with the steel supplier we chose, but the timeline still depends on the delivery date of March 3.

Could you lay out the final numbers in a single table? I'll review it tonight.

Best,
Tomás

Scoring detail

{
  "cleaned_words": 121,
  "similarity_to_source": 0.959,
  "errors": [
    {
      "wrong": "principle concern",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "lets lead",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "compliments the open",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "be discrete",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "less risks",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "on there delivery",
      "wrong_gone": true,
      "fixed_present": false,
      "score": 0.25
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.983,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.7083,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Mellum2.1 12B-A2.5B

✗ fail · score 73 · 2.07 s

Hi Ana,

Thanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:

1. The principle concern from the last meeting was cost, so let's lead with the $412,000 estimate and the phased payment plan.
2. The new lighting plan really complements the open floor layout. I'd keep the renderings on page 6.
3. We need to be discrete about the subcontractor change until the contract is signed.
4. There are less risks with the steel supplier we chose, but the timeline still depends on their delivery date of March 3.

Could you lay out the final numbers in a single table? I'll review it tonight.

Best,
Tomás

Scoring detail

{
  "cleaned_words": 121,
  "similarity_to_source": 0.975,
  "errors": [
    {
      "wrong": "principle concern",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "lets lead",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "compliments the open",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "be discrete",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "less risks",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "on there delivery",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.975,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.5,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

Ling-3.0-tiny (7.9B-A1.3B)

✗ fail · score 82 · 1.56 s

Hi Ana,

Thanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:

1. The principal concern from the last meeting was cost, so let's lead with the $412,000 estimate and the phased payment plan.
2. The new lighting plan really compliments the open floor layout. I'd keep the renderings on page 6.
3. We need to be discrete about the subcontractor change until the contract is signed.
4. There are fewer risks with the steel supplier we chose, but the timeline still depends on their delivery date of March 3.

Could you lay out the final numbers in a single table? I'll review it tonight.

Best,
Tomás

Scoring detail

{
  "cleaned_words": 121,
  "similarity_to_source": 0.967,
  "errors": [
    {
      "wrong": "principle concern",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "lets lead",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "compliments the open",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "be discrete",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "less risks",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "on there delivery",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.983,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 1,
  "checks": {
    "errors": 0.6667,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

MiniCPM5 2B

✗ fail · score 18 · 2.33 s

Hi Ana,

Thanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:

1. The principle concern from the last meeting was cost, so let's lead with the $412,000 estimate and the phased payment plan.
2. The new lighting plan really compliments the open floor layout. I'd keep the renderings on page 6.
3. We need to be discrete about the subcontractor change until the contract is signed.
4. There are less risks with the steel supplier we chose, but the timeline still depends on there delivery date of March 3.

Could you lay out the final numbers in a single table? I'll review it tonight.

Best,
Tomás

Scoring detail

{
  "cleaned_words": 121,
  "similarity_to_source": 0.992,
  "errors": [
    {
      "wrong": "principle concern",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "lets lead",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "compliments the open",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "be discrete",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "less risks",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "on there delivery",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.959,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 0.3333,
  "checks": {
    "errors": 0.1667,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

K2-Horizon 0.9B

✗ fail · score 18 · 0.81 s

Hi Ana,

Thanks for sending over the revised proposal for the Carver Street renovation. A few notes before we present it to the client on Wednesday:

1. The principle concern from the last meeting was cost, so let’s lead with the $412,000 estimate and the phased payment plan.
2. The new lighting plan really compliments the open floor layout. I’d keep the renderings on page 6.
3. We need to be discrete about the subcontractor change until the contract is signed.
4. There are less risks with the steel supplier we chose, but the timeline still depends on there delivery date of March 3.

Could you lay out the final numbers in a single table? I’ll review it tonight.

Best,
Tomás

Scoring detail

{
  "cleaned_words": 121,
  "similarity_to_source": 0.992,
  "errors": [
    {
      "wrong": "principle concern",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "lets lead",
      "wrong_gone": true,
      "fixed_present": true,
      "score": 1
    },
    {
      "wrong": "compliments the open",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "be discrete",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "less risks",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    },
    {
      "wrong": "on there delivery",
      "wrong_gone": false,
      "fixed_present": false,
      "score": 0
    }
  ],
  "fidelity": {
    "similarity_to_reference": 0.959,
    "unknown_words": [],
    "reworded": false,
    "unrequested_edits": []
  },
  "preserve_missing": [],
  "must_keep_missing": [],
  "length_ratio": 1,
  "length_bounds": [
    0.8,
    1.25
  ],
  "gate": 0.3333,
  "checks": {
    "errors": 0.1667,
    "fidelity": 1,
    "preserve": 1,
    "must_keep": 1,
    "length": 1
  }
}

← rewrite-013All Rewrite and proofread itemsrewrite-015 →