Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Translate
translate-025
easy en-zh-hanssign
The prompt
user
Translate the following text into Simplified Chinese. Reply with only the translation. Staff only. No entry.
What we expected
{
"source_lang": "en",
"target_lang": "zh",
"register_note": null,
"source": "Staff only. No entry.",
"reference": "仅限员工，禁止入内。",
"alt_references": [
"员工专用，闲人免进。",
"仅限工作人员，禁止进入。",
"非工作人员禁止入内。"
],
"must_keep": [
[
"员工",
"工作人员"
]
],
"register_check": null,
"avoid": []
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 0.39 s
仅限员工，禁止入内。
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "zh",
"cleaned": "仅限员工，禁止入内。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 100,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 1
}Ornith-1.5 9B
✓ pass · score 99 (scripted checks 99, judges 10.0/10 from 3) · 0.34 s
员工专用，禁止入内。
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "zh",
"cleaned": "员工专用，禁止入内。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 48.81,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 0.9702
}Granite 4.2 8B
✓ pass · score 77 (scripted checks 65, judges 9.0/10 from 3) · 0.47 s
仅限员工。不允许进入。
Judge medians: Adequacy 10 · Fluency and register 8
Scoring detail
{
"target_lang": "zh",
"cleaned": "仅限员工。不允许进入。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 21.62,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 0.2905
}Spark-X2.5 4B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 0.24 s
仅限工作人员。禁止进入。
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "zh",
"cleaned": "仅限工作人员。禁止进入。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 58.47,
"best_ref": 2,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 1
}Mellum2.1 12B-A2.5B
✓ pass · score 89 (scripted checks 78, judges 10.0/10 from 3) · 0.20 s
仅限员工。禁止进入。
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "zh",
"cleaned": "仅限员工。禁止进入。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 32.07,
"best_ref": 2,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 0.5519
}Ling-3.0-tiny (7.9B-A1.3B)
✓ pass · score 98 (scripted checks 95, judges 10.0/10 from 3) · 0.15 s
仅限工作人员。禁止入内。
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "zh",
"cleaned": "仅限工作人员。禁止入内。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 46.12,
"best_ref": 3,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 0.9031
}MiniCPM5 2B
✓ pass · score 93 (scripted checks 95, judges 9.0/10 from 3) · 0.18 s
仅限员工进入。禁止入内。
Judge medians: Adequacy 10 · Fluency and register 8
Scoring detail
{
"target_lang": "zh",
"cleaned": "仅限员工进入。禁止入内。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 46.12,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 0.9031
}K2-Horizon 0.9B
✗ fail · score 55 (scripted checks 66, judges 4.5/10 from 3) · 0.36 s
仅限员工。无入口。
Judge medians: Adequacy 5 · Fluency and register 4
Scoring detail
{
"target_lang": "zh",
"cleaned": "仅限员工。无入口。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 22.61,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 0.3153
}