Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Translate
translate-001
easy en-essign
The prompt
user
Translate the following text into Spanish. Reply with only the translation. Wet floor. Please use the other entrance.
What we expected
{
"source_lang": "en",
"target_lang": "es",
"register_note": null,
"source": "Wet floor. Please use the other entrance.",
"reference": "Piso mojado. Por favor, use la otra entrada.",
"alt_references": [
"Suelo mojado. Por favor, utilice la otra entrada.",
"Cuidado: piso mojado. Use la otra entrada, por favor."
],
"must_keep": [],
"register_check": null,
"avoid": []
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 0.60 s
Suelo mojado. Por favor, utilice la otra entrada.
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "es",
"cleaned": "Suelo mojado. Por favor, utilice la otra entrada.",
"detected_lang": "es",
"lang_ok": true,
"chrf": 100,
"best_ref": 1,
"pass_bar": 30,
"checks_passed": "2/2",
"failed_checks": [],
"chrf_scaled": 1
}Ornith-1.5 9B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 0.50 s
Suelo mojado. Por favor, utilicen la otra entrada.
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "es",
"cleaned": "Suelo mojado. Por favor, utilicen la otra entrada.",
"detected_lang": "es",
"lang_ok": true,
"chrf": 93,
"best_ref": 1,
"pass_bar": 30,
"checks_passed": "2/2",
"failed_checks": [],
"chrf_scaled": 1
}Granite 4.2 8B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 0.52 s
Piso mojado. Por favor, use la otra entrada.
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "es",
"cleaned": "Piso mojado. Por favor, use la otra entrada.",
"detected_lang": "es",
"lang_ok": true,
"chrf": 100,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "2/2",
"failed_checks": [],
"chrf_scaled": 1
}Spark-X2.5 4B
✓ pass · score 98 (scripted checks 100, judges 9.5/10 from 3) · 0.36 s
Zona con suelo mojado. Por favor, use la otra entrada.
Judge medians: Adequacy 10 · Fluency and register 9
Scoring detail
{
"target_lang": "es",
"cleaned": "Zona con suelo mojado. Por favor, use la otra entrada.",
"detected_lang": "es",
"lang_ok": true,
"chrf": 87.66,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "2/2",
"failed_checks": [],
"chrf_scaled": 1
}Mellum2.1 12B-A2.5B
✓ pass · score 98 (scripted checks 100, judges 9.5/10 from 3) · 0.24 s
Piso mojado. Por favor use la otra entrada.
Judge medians: Adequacy 10 · Fluency and register 9
Scoring detail
{
"target_lang": "es",
"cleaned": "Piso mojado. Por favor use la otra entrada.",
"detected_lang": "es",
"lang_ok": true,
"chrf": 90.11,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "2/2",
"failed_checks": [],
"chrf_scaled": 1
}Ling-3.0-tiny (7.9B-A1.3B)
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 0.20 s
Piso mojado. Por favor, use la otra entrada.
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "es",
"cleaned": "Piso mojado. Por favor, use la otra entrada.",
"detected_lang": "es",
"lang_ok": true,
"chrf": 100,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "2/2",
"failed_checks": [],
"chrf_scaled": 1
}MiniCPM5 2B
✓ pass · score 88 (scripted checks 96, judges 8.0/10 from 3) · 0.19 s
Piso mojado. Usa la otra entrada.
Judge medians: Adequacy 8 · Fluency and register 8
Scoring detail
{
"target_lang": "es",
"cleaned": "Piso mojado. Usa la otra entrada.",
"detected_lang": "es",
"lang_ok": true,
"chrf": 61.73,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "2/2",
"failed_checks": [],
"chrf_scaled": 0.9183
}K2-Horizon 0.9B
✗ fail · score 75 (scripted checks 100, judges 5.0/10 from 3) · 0.11 s
Floor mojado. Por favor, use la otra entrada.
Judge medians: Adequacy 5 · Fluency and register 5
Scoring detail
{
"target_lang": "es",
"cleaned": "Floor mojado. Por favor, use la otra entrada.",
"detected_lang": "es",
"lang_ok": true,
"chrf": 88.31,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "2/2",
"failed_checks": [],
"chrf_scaled": 1
}