Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Ask my documents
rag-029
easy answerablesingle_doc
The prompt
Document excerpts provided from: internet_contract, utility_bill
How much is the current electric bill and when is it due?
What we expected
{
"answerable": true,
"answer": "$181.58, due by October 28, 2026.",
"key_facts": [
[
"181.58"
],
[
"october 28",
"oct 28",
"oct. 28",
"10/28",
"10/28",
"28 october",
"28 oct",
"2026-10-28"
]
],
"citations": [
"utility_bill"
],
"contra": [],
"evidence": [
"AMOUNT DUE: $181.58",
"PAYMENT DUE BY: October 28, 2026"
],
"gold_in_context": true
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 · 4.50 s
The current electric bill is $181.58, due by October 28, 2026 [doc:utility_bill].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true,
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"utility_bill"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Ornith-1.5 9B
✓ pass · score 100 · 3.67 s
The current electric bill is $181.58, and it is due by October 28, 2026 [doc:utility_bill].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true,
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"utility_bill"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Granite 4.2 8B
✓ pass · score 100 · 4.28 s
The current electric bill amount is $181.58, and it is due by October 28, 2026. [doc:utility_bill]
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true,
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"utility_bill"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Spark-X2.5 4B
✓ pass · score 100 · 3.15 s
The current electric bill is **$181.58** [doc:utility_bill], and it is due by **October 28, 2026** [doc:utility_bill].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true,
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"utility_bill"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Mellum2.1 12B-A2.5B
✓ pass · score 100 · 3.54 s
The current electric bill is $181.58 and is due on October 28, 2026 [doc:utility_bill].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true,
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"utility_bill"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Ling-3.0-tiny (7.9B-A1.3B)
✓ pass · score 100 · 1.18 s
The current electric bill is $181.58 and it is due by October 28, 2026 [doc:utility_bill].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true,
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"utility_bill"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}MiniCPM5 2B
✓ pass · score 100 · 1.36 s
The current electric bill is $181.58, due by October 28, 2026 [doc:utility_bill].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true,
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"utility_bill"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}K2-Horizon 0.9B
✓ pass · score 100 · 1.03 s
The current electric bill is $181.58 and it is due by October 28, 2026. [doc:utility_bill]
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true,
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"utility_bill"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}