Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Sort and triage

triage-030

medium ticket

The prompt

user

TASK: TICKET

Ticket #48230
From: Henry Osei <henry@oseiconsulting.com>
Plan: Basic
Subject: CSV export puts dates in the wrong column

When I export my paid invoices to CSV, the payment date ends up in the "Client" column and the client name in the "Date" column. I can swap the columns by hand in my spreadsheet so it's not stopping me, but it's annoying and my accountant keeps asking about it.

Henry

Reply with only a JSON object: {"category": "<billing|bug|account_access|feature_request|cancellation>", "priority": "<low|medium|high|urgent>"}

What we expected

{
  "category": "bug",
  "priority": "medium"
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 0.76 s

{"category": "bug", "priority": "medium"}

Scoring detail

{
  "category": {
    "got": "bug",
    "expected": "bug",
    "correct": true
  },
  "priority": {
    "got": "medium",
    "expected": "medium",
    "correct": true
  }
}

Ornith-1.5 9B

✗ fail · score 50 · 0.70 s

{"category": "bug", "priority": "low"}

Scoring detail

{
  "category": {
    "got": "bug",
    "expected": "bug",
    "correct": true
  },
  "priority": {
    "got": "low",
    "expected": "medium",
    "correct": false
  }
}

Granite 4.2 8B

✓ pass · score 100 · 0.74 s

{"category": "bug", "priority": "medium"}

Scoring detail

{
  "category": {
    "got": "bug",
    "expected": "bug",
    "correct": true
  },
  "priority": {
    "got": "medium",
    "expected": "medium",
    "correct": true
  }
}

Spark-X2.5 4B

✓ pass · score 100 · 0.46 s

{"category": "bug", "priority": "medium"}

Scoring detail

{
  "category": {
    "got": "bug",
    "expected": "bug",
    "correct": true
  },
  "priority": {
    "got": "medium",
    "expected": "medium",
    "correct": true
  }
}

Mellum2.1 12B-A2.5B

✓ pass · score 100 · 0.31 s

{"category": "bug", "priority": "medium"}

Scoring detail

{
  "category": {
    "got": "bug",
    "expected": "bug",
    "correct": true
  },
  "priority": {
    "got": "medium",
    "expected": "medium",
    "correct": true
  }
}

Ling-3.0-tiny (7.9B-A1.3B)

✓ pass · score 100 · 0.24 s

{"category": "bug", "priority": "medium"}

Scoring detail

{
  "category": {
    "got": "bug",
    "expected": "bug",
    "correct": true
  },
  "priority": {
    "got": "medium",
    "expected": "medium",
    "correct": true
  }
}

MiniCPM5 2B

✓ pass · score 100 · 0.38 s

{"category": "bug", "priority": "medium"}

Scoring detail

{
  "category": {
    "got": "bug",
    "expected": "bug",
    "correct": true
  },
  "priority": {
    "got": "medium",
    "expected": "medium",
    "correct": true
  }
}

K2-Horizon 0.9B

✓ pass · score 100 · 0.15 s

{"category": "bug", "priority": "medium"}

Scoring detail

{
  "category": {
    "got": "bug",
    "expected": "bug",
    "correct": true
  },
  "priority": {
    "got": "medium",
    "expected": "medium",
    "correct": true
  }
}

← triage-029All Sort and triage itemstriage-031 →