Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Translate

translate-038

medium es-enemail

The prompt

user

Translate the following text into English. Reply with only the translation.

Hola, Sr. Whitfield:

Le escribo para confirmar la reserva de dos habitaciones dobles del 3 al 7 de agosto. El desayuno está incluido, pero el aparcamiento tiene un coste adicional de 12 € por noche. Si necesita la factura a nombre de su empresa, indíquenos el NIF antes de la llegada.

Le recordamos que la entrada es a partir de las 14:00 y la salida, antes de las 12:00. Si llegan más tarde de las 22:00, avísennos con antelación, ya que la recepción cierra a esa hora y les dejaremos las llaves en una caja con código.

Un saludo,
Lucía Ferrer
Hotel Mirador del Puerto

What we expected

{
  "source_lang": "es",
  "target_lang": "en",
  "register_note": null,
  "source": "Hola, Sr. Whitfield:\n\nLe escribo para confirmar la reserva de dos habitaciones dobles del 3 al 7 de agosto. El desayuno está incluido, pero el aparcamiento tiene un coste adicional de 12 € por noche. Si necesita la factura a nombre de su empresa, indíquenos el NIF antes de la llegada.\n\nLe recordamos que la entrada es a partir de las 14:00 y la salida, antes de las 12:00. Si llegan más tarde de las 22:00, avísennos con antelación, ya que la recepción cierra a esa hora y les dejaremos las llaves en una caja con código.\n\nUn saludo,\nLucía Ferrer\nHotel Mirador del Puerto",
  "reference": "Hello Mr. Whitfield,\n\nI am writing to confirm your booking of two double rooms from August 3 to 7. Breakfast is included, but parking costs an extra €12 per night. If you need the invoice in your company's name, please send us your tax ID number before arrival.\n\nPlease note that check-in is from 2:00 p.m. and check-out is by 12:00 noon. If you will arrive after 10:00 p.m., let us know in advance, as reception closes at that time and we will leave the keys for you in a lockbox with a code.\n\nBest regards,\nLucía Ferrer\nHotel Mirador del Puerto",
  "alt_references": [],
  "must_keep": [
    [
      "Whitfield"
    ],
    [
      "two",
      "2"
    ],
    [
      "August"
    ],
    [
      "3 to 7",
      "3 to August 7",
      "3-7",
      "3–7",
      "3rd to",
      "3 through 7",
      "3rd through",
      "3 until 7",
      "3rd until",
      "3rd-7th",
      "3 and 7",
      "3rd and"
    ],
    [
      "€12",
      "12 €",
      "12 euros",
      "EUR 12",
      "12 EUR",
      "€ 12"
    ],
    [
      "14:00",
      "2:00 p.m.",
      "2 p.m.",
      "2 pm",
      "2:00 pm",
      "2pm",
      "2:00pm",
      "2 o'clock"
    ],
    [
      "12:00",
      "noon",
      "midday",
      "12 p.m.",
      "12 pm",
      "12pm"
    ],
    [
      "22:00",
      "10 p.m.",
      "10:00 p.m.",
      "10 pm",
      "10pm",
      "10:00 pm",
      "10:00pm"
    ],
    [
      "Lucía Ferrer",
      "Lucia Ferrer"
    ],
    [
      "Mirador del Puerto"
    ]
  ],
  "register_check": null,
  "avoid": []
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 90 (scripted checks 100, judges 8.0/10 from 3) · 4.39 s

Dear Mr. Whitfield,

We are writing to confirm the reservation of two double rooms from August 3rd to August 7th. Breakfast is included, but parking has an additional cost of €12 per night. If you need the invoice issued in your company's name, please provide us with the NIF before arrival.

Please note that check-in is from 2:00 PM and check-out is before 12:00 PM. If you arrive after 10:00 PM, please notify us in advance, as the reception closes at that time and we will leave your keys in a numbered safe.

Kind regards,
Lucía Ferrer
Hotel Mirador del Puerto

Judge medians: Adequacy 7 · Fluency and register 9

Scoring detail

{
  "target_lang": "en",
  "cleaned": "Dear Mr. Whitfield,\n\nWe are writing to confirm the reservation of two double rooms from August 3rd to August 7th. Breakfast is included, but parking has an additional cost of €12 per night. If you need the invoice issued in your company's name, please provide us with the NIF before arrival.\n\nPlease note that check-in is from 2:00 PM and check-out is before 12:00 PM. If you arrive after 10:00 PM, please notify us in advance, as the reception closes at that time and we will leave your keys in a numbered safe.\n\nKind regards,\nLucía Ferrer\nHotel Mirador del Puerto",
  "detected_lang": "en",
  "lang_ok": true,
  "chrf": 70.91,
  "best_ref": 0,
  "pass_bar": 40,
  "checks_passed": "12/12",
  "failed_checks": [],
  "chrf_scaled": 1
}

Ornith-1.5 9B

✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 3) · 3.73 s

Dear Mr. Whitfield:

I am writing to confirm the reservation of two double rooms from August 3rd to August 7th. Breakfast is included, but parking costs an additional €12 per night. If you need the invoice in your company's name, please let us know the tax identification number before your arrival.

Please be reminded that check-in is from 2:00 PM and check-out is before 12:00 PM. If you arrive after 10:00 PM, please notify us in advance, as the reception closes at that time and we will leave the keys in a code box.

Best regards,
Lucía Ferrer
Hotel Mirador del Puerto

Judge medians: Adequacy 10 · Fluency and register 8

Scoring detail

{
  "target_lang": "en",
  "cleaned": "Dear Mr. Whitfield:\n\nI am writing to confirm the reservation of two double rooms from August 3rd to August 7th. Breakfast is included, but parking costs an additional €12 per night. If you need the invoice in your company's name, please let us know the tax identification number before your arrival.\n\nPlease be reminded that check-in is from 2:00 PM and check-out is before 12:00 PM. If you arrive after 10:00 PM, please notify us in advance, as the reception closes at that time and we will leave the keys in a code box.\n\nBest regards,\nLucía Ferrer\nHotel Mirador del Puerto",
  "detected_lang": "en",
  "lang_ok": true,
  "chrf": 75.41,
  "best_ref": 0,
  "pass_bar": 40,
  "checks_passed": "12/12",
  "failed_checks": [],
  "chrf_scaled": 1
}

Granite 4.2 8B

✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 2) · 3.96 s

Hello Mr. Whitfield:

I am writing to confirm the reservation for two double rooms from August 3 to August 7. Breakfast is included, but parking has an additional cost of 12 € per night. If you need the invoice in your company's name, please let us know your NIF before arrival.

Please note that check-in is from 14:00 and check-out is before 12:00. If you arrive after 22:00, please notify us in advance, as reception closes at that time and we will leave your keys in a locked box with a code.

Best regards,
Lucía Ferrer
Hotel Mirador del Puerto

Judge medians: Adequacy 9.5 · Fluency and register 8.5

Scoring detail

{
  "target_lang": "en",
  "cleaned": "Hello Mr. Whitfield:\n\nI am writing to confirm the reservation for two double rooms from August 3 to August 7. Breakfast is included, but parking has an additional cost of 12 € per night. If you need the invoice in your company's name, please let us know your NIF before arrival.\n\nPlease note that check-in is from 14:00 and check-out is before 12:00. If you arrive after 22:00, please notify us in advance, as reception closes at that time and we will leave your keys in a locked box with a code.\n\nBest regards,\nLucía Ferrer\nHotel Mirador del Puerto",
  "detected_lang": "en",
  "lang_ok": true,
  "chrf": 76.62,
  "best_ref": 0,
  "pass_bar": 40,
  "checks_passed": "12/12",
  "failed_checks": [],
  "chrf_scaled": 1
}

Spark-X2.5 4B

✓ pass · score 93 (scripted checks 100, judges 8.5/10 from 3) · 2.61 s

Hello, Mr. Whitfield:

I am writing to confirm the reservation for two double rooms from August 3 to 7. Breakfast is included, but parking has an additional cost of €12 per night. If you need the invoice in your company's name, please provide us with your NIF before arrival.

We remind you that check-in is from 2:00 PM and check-out is before 12:00 PM. If you arrive after 10:00 PM, please inform us in advance, as the reception closes at that time and we will leave the keys in a code box.

Best regards,
Lucía Ferrer
Hotel Mirador del Puerto

Judge medians: Adequacy 9 · Fluency and register 8

Scoring detail

{
  "target_lang": "en",
  "cleaned": "Hello, Mr. Whitfield:\n\nI am writing to confirm the reservation for two double rooms from August 3 to 7. Breakfast is included, but parking has an additional cost of €12 per night. If you need the invoice in your company's name, please provide us with your NIF before arrival.\n\nWe remind you that check-in is from 2:00 PM and check-out is before 12:00 PM. If you arrive after 10:00 PM, please inform us in advance, as the reception closes at that time and we will leave the keys in a code box.\n\nBest regards,\nLucía Ferrer\nHotel Mirador del Puerto",
  "detected_lang": "en",
  "lang_ok": true,
  "chrf": 73.3,
  "best_ref": 0,
  "pass_bar": 40,
  "checks_passed": "12/12",
  "failed_checks": [],
  "chrf_scaled": 1
}

Mellum2.1 12B-A2.5B

✓ pass · score 93 (scripted checks 100, judges 8.5/10 from 3) · 1.67 s

Hello, Mr. Whitfield:

I am writing to confirm the reservation of two double rooms from August 3 to August 7. Breakfast is included, but parking incurs an additional cost of 12 € per night. If you need the invoice in your company's name, please provide your NIF before arrival.

We remind you that check-in begins at 14:00 and check-out must be before 12:00. If you arrive after 22:00, please let us know in advance, as the reception closes at that time and we will leave the keys in a box with a code.

Best regards,
Lucía Ferrer
Hotel Mirador del Puerto

Judge medians: Adequacy 9 · Fluency and register 8

Scoring detail

{
  "target_lang": "en",
  "cleaned": "Hello, Mr. Whitfield:\n\nI am writing to confirm the reservation of two double rooms from August 3 to August 7. Breakfast is included, but parking incurs an additional cost of 12 € per night. If you need the invoice in your company's name, please provide your NIF before arrival.\n\nWe remind you that check-in begins at 14:00 and check-out must be before 12:00. If you arrive after 22:00, please let us know in advance, as the reception closes at that time and we will leave the keys in a box with a code.\n\nBest regards,\nLucía Ferrer\nHotel Mirador del Puerto",
  "detected_lang": "en",
  "lang_ok": true,
  "chrf": 72.74,
  "best_ref": 0,
  "pass_bar": 40,
  "checks_passed": "12/12",
  "failed_checks": [],
  "chrf_scaled": 1
}

Ling-3.0-tiny (7.9B-A1.3B)

✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 3) · 1.42 s

Hello, Mr. Whitfield:

I am writing to confirm the reservation of two double rooms from August 3rd to August 7th. Breakfast is included, but parking has an additional cost of €12 per night. If you need the invoice in your company's name, please provide the NIF before arrival.

We remind you that check-in begins at 14:00 and check-out is before 12:00. If you arrive after 22:00, please let us know in advance, as the reception closes at that time and we will leave the keys in a code-protected safe.

Best regards,
Lucía Ferrer
Hotel Mirador del Puerto

Judge medians: Adequacy 9 · Fluency and register 9

Scoring detail

{
  "target_lang": "en",
  "cleaned": "Hello, Mr. Whitfield:\n\nI am writing to confirm the reservation of two double rooms from August 3rd to August 7th. Breakfast is included, but parking has an additional cost of €12 per night. If you need the invoice in your company's name, please provide the NIF before arrival.\n\nWe remind you that check-in begins at 14:00 and check-out is before 12:00. If you arrive after 22:00, please let us know in advance, as the reception closes at that time and we will leave the keys in a code-protected safe.\n\nBest regards,\nLucía Ferrer\nHotel Mirador del Puerto",
  "detected_lang": "en",
  "lang_ok": true,
  "chrf": 71.16,
  "best_ref": 0,
  "pass_bar": 40,
  "checks_passed": "12/12",
  "failed_checks": [],
  "chrf_scaled": 1
}

MiniCPM5 2B

✓ pass · score 88 (scripted checks 100, judges 7.5/10 from 3) · 1.65 s

Hello, Mr. Whitfield:

I am writing to confirm the reservation of two double rooms from August 3 to August 7. The breakfast is included, but parking has an additional cost of €12 per night. If you need the invoice on behalf of your company, please indicate the NIF before arrival.

I remind you that the check-in begins at 2:00 PM and check-out, before 12:00. If you arrive later than 10:00 PM, please inform us in advance, as the reception closes at that time and we will leave the keys in a box with a code.

Best regards,
Lucía Ferrer
Hotel Mirador del Puerto

Judge medians: Adequacy 9 · Fluency and register 6

Scoring detail

{
  "target_lang": "en",
  "cleaned": "Hello, Mr. Whitfield:\n\nI am writing to confirm the reservation of two double rooms from August 3 to August 7. The breakfast is included, but parking has an additional cost of €12 per night. If you need the invoice on behalf of your company, please indicate the NIF before arrival.\n\nI remind you that the check-in begins at 2:00 PM and check-out, before 12:00. If you arrive later than 10:00 PM, please inform us in advance, as the reception closes at that time and we will leave the keys in a box with a code.\n\nBest regards,\nLucía Ferrer\nHotel Mirador del Puerto",
  "detected_lang": "en",
  "lang_ok": true,
  "chrf": 66.95,
  "best_ref": 0,
  "pass_bar": 40,
  "checks_passed": "12/12",
  "failed_checks": [],
  "chrf_scaled": 1
}

K2-Horizon 0.9B

✓ pass · score 85 (scripted checks 100, judges 7.0/10 from 3) · 0.68 s

Hello, Mr. Whitfield:

I am writing to confirm the reservation for two double rooms from August 3 to August 7. Breakfast is included, but parking costs an additional 12 € per night. If you need the invoice for your company, please provide your NIF before arrival.

We remind you that the check-in is from 14:00 and the check-out before 12:00. If you arrive after 22:00, please inform us in advance, as reception closes at that time and we will leave the keys in a box with the code.

Best regards,
Lucía Ferrer
Hotel Mirador del Puerto

Judge medians: Adequacy 8 · Fluency and register 6

Scoring detail

{
  "target_lang": "en",
  "cleaned": "Hello, Mr. Whitfield:\n\nI am writing to confirm the reservation for two double rooms from August 3 to August 7. Breakfast is included, but parking costs an additional 12 € per night. If you need the invoice for your company, please provide your NIF before arrival.\n\nWe remind you that the check-in is from 14:00 and the check-out before 12:00. If you arrive after 22:00, please inform us in advance, as reception closes at that time and we will leave the keys in a box with the code.\n\nBest regards,\nLucía Ferrer\nHotel Mirador del Puerto",
  "detected_lang": "en",
  "lang_ok": true,
  "chrf": 69.36,
  "best_ref": 0,
  "pass_bar": 40,
  "checks_passed": "12/12",
  "failed_checks": [],
  "chrf_scaled": 1
}

← translate-037All Translate itemstranslate-039 →