Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Translate
translate-040
hard zh-hans-enproduct
The prompt
user
Translate the following text into English. Reply with only the translation. 本店新推出的手工月饼每盒6枚，每枚约80克，保质期15天。请存放于阴凉干燥处，开封后请尽快食用。中秋节前下单满300元包邮，数量有限，先到先得。
What we expected
{
"source_lang": "zh",
"target_lang": "en",
"register_note": null,
"source": "本店新推出的手工月饼每盒6枚，每枚约80克，保质期15天。请存放于阴凉干燥处，开封后请尽快食用。中秋节前下单满300元包邮，数量有限，先到先得。",
"reference": "Our new handmade mooncakes come 6 to a box, each weighing about 80 g, with a shelf life of 15 days. Store in a cool, dry place and eat soon after opening. Orders of 300 yuan or more placed before the Mid-Autumn Festival ship free. Limited quantities available — first come, first served.",
"alt_references": [],
"must_keep": [
[
"6",
"six"
],
[
"80 g",
"80 grams",
"80g",
"80-gram"
],
[
"15 days",
"fifteen days",
"15-day",
"fifteen-day"
],
[
"300 yuan",
"¥300",
"300 RMB",
"RMB 300",
"CNY 300",
"300 CNY",
"300元",
"RMB300",
"¥ 300"
],
[
"Mid-Autumn",
"Mid Autumn",
"Moon Festival"
],
[
"first come"
]
],
"register_check": null,
"avoid": []
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 2.44 s
Our store's newly released handmade mooncakes come 6 per box, each weighing approximately 80 grams, with a shelf life of 15 days. Please store in a cool, dry place and consume promptly after opening. Orders placed before the Mid-Autumn Festival with a total of 300 yuan or more ship free of charge. Limited quantity, first come first served.
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "en",
"cleaned": "Our store's newly released handmade mooncakes come 6 per box, each weighing approximately 80 grams, with a shelf life of 15 days. Please store in a cool, dry place and consume promptly after opening. Orders placed before the Mid-Autumn Festival with a total of 300 yuan or more ship free of charge. Limited quantity, first come first served.",
"detected_lang": "en",
"lang_ok": true,
"chrf": 73.93,
"best_ref": 0,
"pass_bar": 40,
"checks_passed": "8/8",
"failed_checks": [],
"chrf_scaled": 1
}Ornith-1.5 9B
✓ pass · score 100 (scripted checks 100, judges 10.0/10 from 3) · 2.14 s
Our newly launched handmade mooncakes come in boxes of six, each weighing approximately 80 grams, with a shelf life of 15 days. Please store in a cool, dry place and consume promptly after opening. Orders placed before the Mid-Autumn Festival with a total of 300 yuan or more qualify for free shipping. Limited quantity available on a first-come, first-served basis.
Judge medians: Adequacy 10 · Fluency and register 10
Scoring detail
{
"target_lang": "en",
"cleaned": "Our newly launched handmade mooncakes come in boxes of six, each weighing approximately 80 grams, with a shelf life of 15 days. Please store in a cool, dry place and consume promptly after opening. Orders placed before the Mid-Autumn Festival with a total of 300 yuan or more qualify for free shipping. Limited quantity available on a first-come, first-served basis.",
"detected_lang": "en",
"lang_ok": true,
"chrf": 72.28,
"best_ref": 0,
"pass_bar": 40,
"checks_passed": "8/8",
"failed_checks": [],
"chrf_scaled": 1
}Granite 4.2 8B
✓ pass · score 92 (scripted checks 94, judges 9.0/10 from 3) · 2.15 s
This shop has newly launched handmade mooncakes, six pieces per box, approximately 80 grams each, with a shelf life of 15 days. Please store in a cool, dry place. After opening, consume as soon as possible. Orders placed before Mid-Autumn Festival are eligible for free shipping when reaching 300 yuan; limited quantity, first come first served.
Judge medians: Adequacy 10 · Fluency and register 8
Scoring detail
{
"target_lang": "en",
"cleaned": "This shop has newly launched handmade mooncakes, six pieces per box, approximately 80 grams each, with a shelf life of 15 days. Please store in a cool, dry place. After opening, consume as soon as possible. Orders placed before Mid-Autumn Festival are eligible for free shipping when reaching 300 yuan; limited quantity, first come first served.",
"detected_lang": "en",
"lang_ok": true,
"chrf": 60.39,
"best_ref": 0,
"pass_bar": 40,
"checks_passed": "8/8",
"failed_checks": [],
"chrf_scaled": 0.8847
}Spark-X2.5 4B
✓ pass · score 90 (scripted checks 100, judges 8.0/10 from 3) · 1.71 s
The handmade mooncakes newly launched by our store are 6 pieces per box, each weighing approximately 80 grams, with a shelf life of 15 days. Please store them in a cool, dry place, and consume them as soon as possible after opening. Place an order before the Mid-Autumn Festival for orders of 300 yuan or more, free shipping is included. Quantities are limited, and availability is first-come, first-served.
Judge medians: Adequacy 10 · Fluency and register 6
Scoring detail
{
"target_lang": "en",
"cleaned": "The handmade mooncakes newly launched by our store are 6 pieces per box, each weighing approximately 80 grams, with a shelf life of 15 days. Please store them in a cool, dry place, and consume them as soon as possible after opening. Place an order before the Mid-Autumn Festival for orders of 300 yuan or more, free shipping is included. Quantities are limited, and availability is first-come, first-served.",
"detected_lang": "en",
"lang_ok": true,
"chrf": 65.32,
"best_ref": 0,
"pass_bar": 40,
"checks_passed": "8/8",
"failed_checks": [],
"chrf_scaled": 1
}Mellum2.1 12B-A2.5B
✓ pass · score 90 (scripted checks 99, judges 8.0/10 from 3) · 0.89 s
This shop's newly launched handmade mooncakes are 6 pieces per box, each about 80 grams, with a shelf life of 15 days. Please store in a cool, dry place; after opening, please consume as soon as possible. Order before the Mid-Autumn Festival for free shipping on orders over 300 yuan, limited quantity, first come first served.
Judge medians: Adequacy 9 · Fluency and register 7
Scoring detail
{
"target_lang": "en",
"cleaned": "This shop's newly launched handmade mooncakes are 6 pieces per box, each about 80 grams, with a shelf life of 15 days. Please store in a cool, dry place; after opening, please consume as soon as possible. Order before the Mid-Autumn Festival for free shipping on orders over 300 yuan, limited quantity, first come first served.",
"detected_lang": "en",
"lang_ok": true,
"chrf": 64.35,
"best_ref": 0,
"pass_bar": 40,
"checks_passed": "8/8",
"failed_checks": [],
"chrf_scaled": 0.9838
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 74 (scripted checks 87, judges 6.0/10 from 3) · 0.89 s
Our store's newly launched handmade mooncakes come in boxes of 6 pieces, each weighing approximately 80 grams, with a shelf life of 15 days. Please store them in a cool and dry place, and consume as soon as possible after opening. Place orders for the Mid-Autumn Festival in advance for more than 300 yuan, which will be shipped free. The quantity is limited, and the first to order will be served.
Judge medians: Adequacy 6 · Fluency and register 6
Scoring detail
{
"target_lang": "en",
"cleaned": "Our store's newly launched handmade mooncakes come in boxes of 6 pieces, each weighing approximately 80 grams, with a shelf life of 15 days. Please store them in a cool and dry place, and consume as soon as possible after opening. Place orders for the Mid-Autumn Festival in advance for more than 300 yuan, which will be shipped free. The quantity is limited, and the first to order will be served.",
"detected_lang": "en",
"lang_ok": true,
"chrf": 57.81,
"best_ref": 0,
"pass_bar": 40,
"checks_passed": "7/8",
"failed_checks": [
{
"type": "keep",
"terms": [
"first come"
],
"ok": false,
"found": null
}
],
"chrf_scaled": 0.8202
}MiniCPM5 2B
✗ fail · score 79 (scripted checks 98, judges 6.0/10 from 3) · 0.93 s
Our newly introduced handmade mooncakes, each box contains 6 pieces, each weighing about 80 grams, with a shelf life of 15 days. Please store them in a cool and dry place, and consume them promptly after opening. Orders for mid-Autumn Festival are eligible for free shipping if the total amount is over 300 yuan, and quantities are limited, first come, first served.
Judge medians: Adequacy 6 · Fluency and register 6
Scoring detail
{
"target_lang": "en",
"cleaned": "Our newly introduced handmade mooncakes, each box contains 6 pieces, each weighing about 80 grams, with a shelf life of 15 days. Please store them in a cool and dry place, and consume them promptly after opening. Orders for mid-Autumn Festival are eligible for free shipping if the total amount is over 300 yuan, and quantities are limited, first come, first served.",
"detected_lang": "en",
"lang_ok": true,
"chrf": 63.78,
"best_ref": 0,
"pass_bar": 40,
"checks_passed": "8/8",
"failed_checks": [],
"chrf_scaled": 0.9695
}K2-Horizon 0.9B
✗ fail · score 69 (scripted checks 92, judges 4.5/10 from 3) · 0.45 s
New handmade pastries are being launched, each box contains 6 pieces, each weighing about 80 grams, with a shelf life of 15 days. Please store them in a cool, dry place, and use them as soon as possible after opening. Orders for the Mid-Autumn Festival will be available for purchase at a discount, with limited quantities, and first-come, first-served.
Judge medians: Adequacy 3 · Fluency and register 6
Scoring detail
{
"target_lang": "en",
"cleaned": "New handmade pastries are being launched, each box contains 6 pieces, each weighing about 80 grams, with a shelf life of 15 days. Please store them in a cool, dry place, and use them as soon as possible after opening. Orders for the Mid-Autumn Festival will be available for purchase at a discount, with limited quantities, and first-come, first-served.",
"detected_lang": "en",
"lang_ok": true,
"chrf": 61.85,
"best_ref": 0,
"pass_bar": 40,
"checks_passed": "7/8",
"failed_checks": [
{
"type": "keep",
"terms": [
"300 yuan",
"¥300",
"300 RMB",
"RMB 300",
"CNY 300",
"300 CNY",
"300元",
"RMB300",
"¥ 300"
],
"ok": false,
"found": null
}
],
"chrf_scaled": 0.9214
}