Unified MemoryBenchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Ask my documents
rag-001
easy answerablesingle_doc
The prompt
Document excerpts provided from: lease, utility_bill
If our rent is paid after the grace period, what late fee does the lease charge?
What we expected
{
"answerable": true,
"answer": "$75, plus $5 per day after that (late charges capped at $150 a month).",
"key_facts": [
[
"$75",
"75 dollars"
]
],
"citations": [
"lease"
],
"contra": [],
"evidence": [
"late fee of $75.00"
],
"gold_in_context": true
}What each model answered
MiMo-V2.6 Distill 9B
✓ pass · score 100 · 4.93 s
The lease charges a **$75.00 late fee**, plus **$5.00 per day** for each additional day rent remains unpaid, with total late charges capped at **$150.00** per month [doc:lease].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"lease"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Ornith-1.5 9B
✓ pass · score 100 · 4.25 s
The lease charges a late fee of **$75.00**, plus **$5.00 per day** for each additional day rent remains unpaid, with total late charges not exceeding **$150.00** in any one month [doc:lease].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"lease"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Granite 4.2 8B
✓ pass · score 100 · 6.56 s
[doc:lease] indicates that rent not received by 11:59 p.m. on the 5th day of the month is late, and the tenant will be charged a late fee of $75.00 plus $5.00 per day for each additional day rent remains unpaid, with total late charges not exceeding $150.00 in any one month. Thus, if rent is paid after the grace period (i.e., after the 5th), the lease charges a $75.00 late fee plus $5.00 per day for each additional day beyond the first late day, up to a maximum of $150.00 in the month.
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"lease"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Spark-X2.5 4B
✓ pass · score 100 · 2.81 s
If rent is not paid by the 5th day of the month, the lease charges a late fee of **$75.00** [doc:lease].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"lease"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Mellum2.1 12B-A2.5B
✗ fail · score 30 · 9.99 s
[doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease] [doc:lease]
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
false
],
"contra_hit": [],
"coverage": 0,
"cited": [
"lease"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}Ling-3.0-tiny (7.9B-A1.3B)
✓ pass · score 100 · 1.22 s
The lease charges a late fee of $75.00, plus $5.00 per day for each additional day, with a maximum total of $150.00 per month [doc:lease].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"lease"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}MiniCPM5 2B
✓ pass · score 100 · 1.65 s
The lease states that rent not received by 11:59 p.m. on the 5th day of the month is late, and the late fee is $75.00, plus $5.00 per day for each additional day rent remains unpaid [doc:lease].
Scoring detail
{
"answerable": true,
"refusal": false,
"facts_matched": [
true
],
"contra_hit": [],
"coverage": 1,
"cited": [
"lease"
],
"citation_recall": 1,
"extra_citations": [],
"citation_score": 1
}K2-Horizon 0.9B
✗ fail · score 0 · 1.58 s
I can't find that in your documents.
Scoring detail
{
"answerable": true,
"refusal": true,
"facts_matched": [
false
],
"contra_hit": [],
"coverage": 0,
"cited": [],
"citation_recall": 0,
"extra_citations": [],
"citation_score": 0
}