Unified Memory

Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Sort and triage

triage-026

easy ticket

The prompt

user

TASK: TICKET

Ticket #48207
From: Jamal Ortiz <jamal@ortizlandscaping.net>
Plan: Pro
Subject: Suggestion: colored logo on invoice emails

Love the app. One thing I'd like to see someday: the option to show my logo in full color at the top of the invoice emails, instead of just the black and white version. Not a big deal, just an idea!

Jamal

Reply with only a JSON object: {"category": "<billing|bug|account_access|feature_request|cancellation>", "priority": "<low|medium|high|urgent>"}
What we expected
{
  "category": "feature_request",
  "priority": "low"
}

What each model answered

MiMo-V2.6 Distill 9B

✓ pass · score 100 · 0.75 s

{"category": "feature_request", "priority": "low"}
Scoring detail
{
  "category": {
    "got": "feature_request",
    "expected": "feature_request",
    "correct": true
  },
  "priority": {
    "got": "low",
    "expected": "low",
    "correct": true
  }
}

Ornith-1.5 9B

✓ pass · score 100 · 0.69 s

{"category": "feature_request", "priority": "low"}
Scoring detail
{
  "category": {
    "got": "feature_request",
    "expected": "feature_request",
    "correct": true
  },
  "priority": {
    "got": "low",
    "expected": "low",
    "correct": true
  }
}

Granite 4.2 8B

✓ pass · score 100 · 0.80 s

{"category": "feature_request", "priority": "low"}
Scoring detail
{
  "category": {
    "got": "feature_request",
    "expected": "feature_request",
    "correct": true
  },
  "priority": {
    "got": "low",
    "expected": "low",
    "correct": true
  }
}

Spark-X2.5 4B

✓ pass · score 100 · 0.45 s

{"category": "feature_request", "priority": "low"}
Scoring detail
{
  "category": {
    "got": "feature_request",
    "expected": "feature_request",
    "correct": true
  },
  "priority": {
    "got": "low",
    "expected": "low",
    "correct": true
  }
}

Mellum2.1 12B-A2.5B

✓ pass · score 100 · 0.29 s

{"category":"feature_request","priority":"low"}
Scoring detail
{
  "category": {
    "got": "feature_request",
    "expected": "feature_request",
    "correct": true
  },
  "priority": {
    "got": "low",
    "expected": "low",
    "correct": true
  }
}

Ling-3.0-tiny (7.9B-A1.3B)

✓ pass · score 100 · 0.25 s

{"category": "feature_request", "priority": "low"}
Scoring detail
{
  "category": {
    "got": "feature_request",
    "expected": "feature_request",
    "correct": true
  },
  "priority": {
    "got": "low",
    "expected": "low",
    "correct": true
  }
}

MiniCPM5 2B

✓ pass · score 100 · 0.38 s

{"category": "feature_request", "priority": "low"}
Scoring detail
{
  "category": {
    "got": "feature_request",
    "expected": "feature_request",
    "correct": true
  },
  "priority": {
    "got": "low",
    "expected": "low",
    "correct": true
  }
}

K2-Horizon 0.9B

✓ pass · score 100 · 0.15 s

{"category": "feature_request", "priority": "low"}
Scoring detail
{
  "category": {
    "got": "feature_request",
    "expected": "feature_request",
    "correct": true
  },
  "priority": {
    "got": "low",
    "expected": "low",
    "correct": true
  }
}

← triage-025All Sort and triage itemstriage-027 →