Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Ask my documents
rag-031
hard answerablesingle_doc
The prompt
Document excerpts provided from: afterschool_program, gym_membership, sitter_notes
With the sibling discount, what is our total monthly Kids Club after-school tuition for Theo and Maya?
What we expected
{
"answerable": true,
"answer": "$503.50: Theo's 5-day plan $310 plus Maya's 3-day plan $215 less 10% ($193.50).",
"key_facts": [
[
"503.50",
"503.5"
]
],
"citations": [
"afterschool_program"
],
"contra": [],
"evidence": [
"| 5-day (Mon–Fri) | $310 |",
"| 3-day (Mon/Wed/Fri) | $215 |",
"10% off the tuition of the second enrolled child",
"applied to the lower of the two tuitions"
],
"gold_in_context": true
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 · 5.59 s
Theo is on the 5-day plan ($310) and Maya is on the 3-day plan ($215). The 10% sibling discount applies to the lower tuition, which is Maya's $215, so the discount is $21.50. Total monthly tuition is $310 + $215 − $21.50 = $503.50. [doc:afterschool_program]
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"afterschool_program"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Ornith-1.5 9B
✓ pass · score 100 · 4.63 s
Theo is on the 5-day plan ($310) and Maya is on the 3-day plan ($215). The sibling discount is 10% off the lower tuition, so 10% of $215 = $21.50 off. Total monthly tuition = $310 + $215 − $21.50 = **$503.50**. [doc:afterschool_program]
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"afterschool_program"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Granite 4.2 8B
✓ pass · score 100 · 6.46 s
[1] shows the tuition plans: 5-day ($310) for Theo and 3-day ($215) for Maya. [1] states the sibling discount is 10% off the tuition of the second enrolled child, applied to the lower of the two tuitions. Thus, Maya’s lower tuition ($215) receives a 10% discount ($21.50), reducing her tuition to $193.50. The total monthly tuition for Theo and Maya is $310 + $193.50 = $503.50. [doc:afterschool_program] and [doc:afterschool_program] provide the tuition amounts and discount rule.
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"afterschool_program"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Spark-X2.5 4B
✓ pass · score 100 · 6.93 s
To calculate the total monthly Kids Club after-school tuition for Theo and Maya with the sibling discount, we first identify their plans and tuition from [doc:afterschool_program] and [doc:afterschool_program] [2].
- **Theo (Grade 1):** 5-day plan, $310/month.
- **Maya (Grade 4):** 3-day plan (Mon/Wed/Fri), $215/month.
The sibling discount is 10% off the tuition of the second enrolled child, applied to the lower of the two tuitions [doc:afterschool_program].
1. **Identify the lower tuition:** Maya's 3-day plan ($215) is lower than Theo's 5-day plan ($310).
2. **Apply the discount:** 10% off Maya's tuition.
$215 * (1 - 0.10) = $215 * 0.90 = $193.50.
3. **Calculate the total:** Add Theo's tuition and the discounted Maya's tuition.
$310 (Theo) + $193.50 (Maya) = **$503.50**.
The total monthly Kids Club after-school tuition for Theo and Maya with the sibling discount is **$503.50** [doc:afterschool_program] [doc:afterschool_program] [2].Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"afterschool_program"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Mellum2.1 12B-A2.5B
✓ pass · score 100 · 5.66 s
[doc:afterschool_program] shows the tuition for Theo (5-day plan $310) and Maya (3-day plan $215). The sibling discount is 10% off the tuition of the second enrolled child, applied to the lower of the two tuitions. The lower tuition is $215, so 10% of $215 is $21.50 discount. Total before discount: $310 + $215 = $525. Discount: $21.50, so total = $503.50. However, we need to ensure the discount is applied correctly: "10% off the tuition of the second enrolled child. The discount is applied to the lower of the two tuitions." The second enrolled child's tuition is the lower one? Actually "second enrolled child" could be the second child enrolled, but here we have two children: Theo (5-day) and Maya (3-day). The discount is 10% off the tuition of the second enrolled child, applied to the lower of the two tuitions. So the discount is 10% of the lower tuition ($215) = $21.50. So total = $310 + $215 - $21.50 = $503.50. We have the numbers in [doc:afterschool_program] and [doc:afterschool_program] again (same). So answer: $503.50. But we need to cite documents: [doc:afterschool_program] for tuition rates and discount rule. Also [doc:afterschool_program] again maybe same. We'll cite once. So answer: $503.50. Keep short one to three sentences. Let's produce: The total monthly tuition for Theo ($310) and Maya ($215) with the sibling discount (10% off the lower tuition $215) is $503.50 [doc:afterschool_program].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"afterschool_program"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Ling-3.0-tiny (7.9B-A1.3B)
✗ fail · score 0 · 1.45 s
The sibling discount is 10% off the lower tuition [doc:1]. For Theo (5-day, $310) and Maya (3-day, $215), the lower is $215. The discounted tuition is $215 × 0.90 = $193.50. I can't find that in your documents.
Scoring detail
{
"answerable": true,
"refusal": true,
"facts_matched": [
false
],
"contra_hit": [],
"coverage": 0,
"cited": [
"1"
],
"citation_recall": 0,
"extra_citations": [
"1"
],
"citation_score": 0
}MiniCPM5 2B
✓ pass · score 100 · 1.81 s
Theo's 5-day plan costs $310 per month [doc:afterschool_program]. With the sibling discount, Maya's 3-day plan of $215 gets 10% off, but the discount applies to the lower tuition. The total monthly tuition for both is $310 + ($215 × 0.90) = $310 + $193.50 = $503.50 [doc:afterschool_program].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"afterschool_program"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}K2-Horizon 0.9B
✗ fail · score 30 · 0.52 s
[doc:afterschool_program]
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
false
],
"contra_hit": [],
"coverage": 0,
"cited": [
"afterschool_program"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}