Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Translate
translate-028
medium en-zh-hansrecipe
The prompt
user
Translate the following text into Simplified Chinese. Reply with only the translation. Soak the rice for 30 minutes, then rinse it until the water runs clear. Use 1.2 cups of water for every cup of rice. Bring to a boil, cover, and simmer on low heat for 15 minutes.
What we expected
{
"source_lang": "en",
"target_lang": "zh",
"register_note": null,
"source": "Soak the rice for 30 minutes, then rinse it until the water runs clear. Use 1.2 cups of water for every cup of rice. Bring to a boil, cover, and simmer on low heat for 15 minutes.",
"reference": "将米浸泡30分钟，然后淘洗至水变清。每杯米加1.2杯水。煮沸后盖上锅盖，转小火焖煮15分钟。",
"alt_references": [],
"must_keep": [
[
"30分钟",
"三十分钟",
"半小时",
"半个小时"
],
[
"1.2"
],
[
"15分钟",
"十五分钟",
"一刻钟"
],
[
"小火",
"慢火",
"文火",
"微火"
]
],
"register_check": null,
"avoid": []
}What each model answered
MiMo-V2.6 Distill 9B
✗ fail · score 68 (scripted checks 81, judges 5.5/10 from 3) · 3.62 s
将大米浸泡30分钟，然后冲洗至水变清澈。每杯米使用1.2杯水。煮沸后盖上锅盖，用小火 simmer 15分钟。 （注：原文"simmer"为英文，已按上下文译为"小火慢煮"，但为保持原文用词，此处保留英文。如需完全中文，可改为"小火慢煮15分钟"。） 最终翻译：将大米浸泡30分钟，然后冲洗至水变清澈。每杯米使用1.2杯水。煮沸后盖上锅盖，用小火慢煮15分钟。
Judge medians: Adequacy 7 · Fluency and register 4
Scoring detail
{
"target_lang": "zh",
"cleaned": "将大米浸泡30分钟，然后冲洗至水变清澈。每杯米使用1.2杯水。煮沸后盖上锅盖，用小火 simmer 15分钟。\n\n（注：原文\"simmer\"为英文，已按上下文译为\"小火慢煮\"，但为保持原文用词，此处保留英文。如需完全中文，可改为\"小火慢煮15分钟\"。）\n\n最终翻译：将大米浸泡30分钟，然后冲洗至水变清澈。每杯米使用1.2杯水。煮沸后盖上锅盖，用小火慢煮15分钟。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 39.04,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "5/6",
"failed_checks": [
{
"type": "length",
"ratio": 3.826,
"ok": false
}
],
"chrf_scaled": 0.7259
}Ornith-1.5 9B
✓ pass · score 83 (scripted checks 100, judges 6.5/10 from 3) · 1.16 s
将米饭浸泡30分钟，然后冲洗至水变清澈。每杯米饭使用1.2杯水的量。煮沸后盖上锅盖，用小火焖煮15分钟。
Judge medians: Adequacy 7 · Fluency and register 6
Scoring detail
{
"target_lang": "zh",
"cleaned": "将米饭浸泡30分钟，然后冲洗至水变清澈。每杯米饭使用1.2杯水的量。煮沸后盖上锅盖，用小火焖煮15分钟。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 58.83,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "6/6",
"failed_checks": [],
"chrf_scaled": 1
}Granite 4.2 8B
✗ fail · score 69 (scripted checks 79, judges 6.0/10 from 3) · 1.53 s
浸泡米饭30分钟，然后冲洗至水清澈。每1杯米用1.2杯水。煮沸，盖上锅，在低温下炖15分钟。
Judge medians: Adequacy 7 · Fluency and register 5
Scoring detail
{
"target_lang": "zh",
"cleaned": "浸泡米饭30分钟，然后冲洗至水清澈。每1杯米用1.2杯水。煮沸，盖上锅，在低温下炖15分钟。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 37.1,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "5/6",
"failed_checks": [
{
"type": "keep",
"terms": [
"小火",
"慢火",
"文火",
"微火"
],
"ok": false,
"found": null
}
],
"chrf_scaled": 0.6775
}Spark-X2.5 4B
✓ pass · score 98 (scripted checks 100, judges 9.5/10 from 3) · 0.77 s
将米浸泡30分钟，然后淘洗至水清澈。每杯米使用1.2杯水。煮沸后盖上锅盖，用小火慢炖15分钟。
Judge medians: Adequacy 10 · Fluency and register 9
Scoring detail
{
"target_lang": "zh",
"cleaned": "将米浸泡30分钟，然后淘洗至水清澈。每杯米使用1.2杯水。煮沸后盖上锅盖，用小火慢炖15分钟。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 65.41,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "6/6",
"failed_checks": [],
"chrf_scaled": 1
}Mellum2.1 12B-A2.5B
✗ fail · score 63 (scripted checks 81, judges 4.5/10 from 3) · 0.70 s
将米子浸泡30分钟，然后用清水冲洗直到水流清澈。每杯米子使用1.2杯水。水开后煮沸，盖上锅盖，低火 simmer 15分钟。
Judge medians: Adequacy 5 · Fluency and register 4
Scoring detail
{
"target_lang": "zh",
"cleaned": "将米子浸泡30分钟，然后用清水冲洗直到水流清澈。每杯米子使用1.2杯水。水开后煮沸，盖上锅盖，低火 simmer 15分钟。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 39.16,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "5/6",
"failed_checks": [
{
"type": "keep",
"terms": [
"小火",
"慢火",
"文火",
"微火"
],
"ok": false,
"found": null
}
],
"chrf_scaled": 0.7289
}Ling-3.0-tiny (7.9B-A1.3B)
✓ pass · score 92 (scripted checks 94, judges 9.0/10 from 3) · 0.39 s
将大米浸泡30分钟，然后冲洗直到水变清澈。每杯大米使用1.2杯水。将水煮沸，盖上锅盖，用小火炖煮15分钟。
Judge medians: Adequacy 10 · Fluency and register 8
Scoring detail
{
"target_lang": "zh",
"cleaned": "将大米浸泡30分钟，然后冲洗直到水变清澈。每杯大米使用1.2杯水。将水煮沸，盖上锅盖，用小火炖煮15分钟。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 44.8,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "6/6",
"failed_checks": [],
"chrf_scaled": 0.8699
}MiniCPM5 2B
✓ pass · score 86 (scripted checks 86, judges 8.5/10 from 3) · 0.47 s
将米浸泡30分钟，然后冲洗直到水变清澈。使用每杯米1.2杯的水。烧开后盖住，慢火煮15分钟。
Judge medians: Adequacy 10 · Fluency and register 7
Scoring detail
{
"target_lang": "zh",
"cleaned": "将米浸泡30分钟，然后冲洗直到水变清澈。使用每杯米1.2杯的水。烧开后盖住，慢火煮15分钟。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 39.08,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "6/6",
"failed_checks": [],
"chrf_scaled": 0.7269
}K2-Horizon 0.9B
✗ fail · score 60 (scripted checks 65, judges 5.5/10 from 3) · 0.35 s
浸泡米饭30分钟后，用水冲洗，直到水清澈。用1.2杯水对每杯米饭加水。将米饭加入水中，煮沸，盖上，低温煮15分钟。
Judge medians: Adequacy 6 · Fluency and register 5
Scoring detail
{
"target_lang": "zh",
"cleaned": "浸泡米饭30分钟后，用水冲洗，直到水清澈。用1.2杯水对每杯米饭加水。将米饭加入水中，煮沸，盖上，低温煮15分钟。",
"detected_lang": "zh",
"lang_ok": true,
"chrf": 26.15,
"best_ref": 0,
"pass_bar": 30,
"checks_passed": "5/6",
"failed_checks": [
{
"type": "keep",
"terms": [
"小火",
"慢火",
"文火",
"微火"
],
"ok": false,
"found": null
}
],
"chrf_scaled": 0.4037
}