Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Translate
translate-026
easy en-zh-hanschat
The prompt
user
Translate the following text into Simplified Chinese. Reply with only the translation. Did you get home okay? Text me when you're there.
What we expected
{
"source_lang": "en",
"target_lang": "zh",
"register_note": null,
"source": "Did you get home okay? Text me when you're there.",
"reference": "你安全到家了吗？到了给我发个消息。",
"alt_references": [
"你平安到家了吗？到了给我发个短信。",
"到家了吗？到了跟我说一声。",
"你到家了没？到了发个消息给我。"
],
"must_keep": [
[
"到家",
"回家"
]
],
"register_check": null,
"avoid": []
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 95 (scripted checks 100, judges 9.0/10 from 3) · 0.50 s
你到家了吗？到了给我发个消息。
Judge medians: Adequacy 8 · Fluency and register 10
Scoring detail
{
"target_lang": "zh",
"cleaned": "你到家了吗？到了给我发个消息。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 82.29,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 1
}Ornith-1.5 9B
✓ pass · score 98 (scripted checks 100, judges 9.5/10 from 3) · 0.48 s
你到家了吗？到了给我发条信息。
Judge medians: Adequacy 9 · Fluency and register 10
Scoring detail
{
"target_lang": "zh",
"cleaned": "你到家了吗？到了给我发条信息。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 56.57,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 1
}Granite 4.2 8B
✗ fail · score 57 (scripted checks 63, judges 5.0/10 from 3) · 0.54 s
你回家没事吗？到了时候发我消息。
Judge medians: Adequacy 6 · Fluency and register 4
Scoring detail
{
"target_lang": "zh",
"cleaned": "你回家没事吗？到了时候发我消息。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 20.77,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 0.2693
}Spark-X2.5 4B
✓ pass · score 73 (scripted checks 72, judges 7.5/10 from 3) · 0.34 s
你到家还好吗？到了之后给我发个信息。
Judge medians: Adequacy 8 · Fluency and register 7
Scoring detail
{
"target_lang": "zh",
"cleaned": "你到家还好吗？到了之后给我发个信息。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 27.56,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 0.439
}Mellum2.1 12B-A2.5B
✓ pass · score 68 (scripted checks 70, judges 6.5/10 from 3) · 0.24 s
你回家了吗？到家后发我。
Judge medians: Adequacy 7 · Fluency and register 6
Scoring detail
{
"target_lang": "zh",
"cleaned": "你回家了吗？到家后发我。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 26.1,
"best_ref": 2,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 0.4026
}Ling-3.0-tiny (7.9B-A1.3B)
✓ pass · score 98 (scripted checks 100, judges 9.5/10 from 3) · 0.17 s
你到家了没？到了就给我发消息。
Judge medians: Adequacy 9 · Fluency and register 10
Scoring detail
{
"target_lang": "zh",
"cleaned": "你到家了没？到了就给我发消息。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 51.97,
"best_ref": 3,
"pass_bar": 20,
"checks_passed": "3/3",
"failed_checks": [],
"chrf_scaled": 1
}MiniCPM5 2B
✗ fail · score 67 (scripted checks 60, judges 7.5/10 from 3) · 0.19 s
你到了吗？到了就联系我。
Judge medians: Adequacy 6 · Fluency and register 9
Scoring detail
{
"target_lang": "zh",
"cleaned": "你到了吗？到了就联系我。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 25.99,
"best_ref": 2,
"pass_bar": 20,
"checks_passed": "2/3",
"failed_checks": [
{
"type": "keep",
"terms": [
"到家",
"回家"
],
"ok": false,
"found": null
}
],
"chrf_scaled": 0.3998
}K2-Horizon 0.9B
✗ fail · score 40 (scripted checks 45, judges 3.5/10 from 3) · 0.12 s
你好吗？如果你回来了，我会给你发消息。
Judge medians: Adequacy 1 · Fluency and register 6
Scoring detail
{
"target_lang": "zh",
"cleaned": "你好吗？如果你回来了，我会给你发消息。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 13.71,
"best_ref": 0,
"pass_bar": 20,
"checks_passed": "2/3",
"failed_checks": [
{
"type": "keep",
"terms": [
"到家",
"回家"
],
"ok": false,
"found": null
}
],
"chrf_scaled": 0.0927
}