Benchmarks / Which Small AI Model Should You Run on a Mac? We Ran 312 Tests / Summarize + extract

summarize-extract-042

medium summarizationarticle

The prompt

user

Summarize the science news article below in at most 90 words. Reply with only the summary.

"""
Smart thermostats save less than advertised, large study finds - but how you use them matters

Smart thermostats are often sold with the promise of cutting heating bills by up to 23%. A new study of more than a thousand homes suggests the real-world savings are much smaller for most people - but not for everyone.

Researchers at the University of Northfield's Energy Behavior Lab tracked 1,150 households in the upper Midwest for two full winters, comparing their gas use before and after installing a smart thermostat. The team adjusted for weather, home size and changes in who lived in each house. The study appears this month in the Journal of Residential Energy.

On average, households cut their heating bills by 8%, or about $96 a year at current gas prices. That's real money, the authors note, but about a third of what manufacturers commonly advertise.

The averages hide a big split. Households that set up and kept a schedule - lowering the temperature at night and while the house was empty - saved an average of 15%. Households that mostly controlled the thermostat by hand, or that turned off the schedule, saved just 2%, which the researchers say is within the margin of error.

And many people did turn the schedule off. About 30% of households overrode or disabled the automatic settings within the first month, most often because a family member found the house too cold in the morning. "People don't give up on saving energy; they give up on being uncomfortable," said lead author Dr. Esther Nwankwo. "The first cold morning decides whether the device pays off."

The researchers found that a simple fix helped: households that set the morning warm-up to start 30 minutes earlier were far less likely to abandon their schedules. The team suggests manufacturers make that the default.

Installation cost matters too. With a typical price of about $180 for the device and $0 to $120 for installation, a household saving the average 8% would take roughly two to three years to break even. A household saving 15% would break even in about a year and a half.

The study has limits. All the homes were in cold-climate states and heated with natural gas, so the results may not apply to homes in warmer areas, those with heat pumps, or to air-conditioning savings in summer. The study was funded by a state energy office; no thermostat manufacturer was involved.

Nwankwo said the takeaway for consumers is straightforward: "If you buy one, spend 20 minutes setting a schedule you can live with, and don't let the first chilly morning talk you out of it."
"""

{
  "kind": "summarize",
  "word_limit": 90,
  "facts": [
    {
      "id": "avg",
      "desc": "average savings 8% (about $96/yr)",
      "any": [
        "\\b8\\s*%|\\b8 percent|eight percent|\\$?96"
      ]
    },
    {
      "id": "claim",
      "desc": "much less than the up-to-23% advertised",
      "any": [
        "23|advertis|claim|promis|a third|one-third|one third|marketed"
      ]
    },
    {
      "id": "sched",
      "desc": "households keeping a schedule saved 15%",
      "any": [
        "15\\s*%|15 percent|fifteen percent"
      ]
    },
    {
      "id": "manual",
      "desc": "manual/override users saved only 2%",
      "any": [
        "\\b2\\s*%|2 percent|two percent|barely|almost nothing|negligible"
      ]
    },
    {
      "id": "override",
      "desc": "about 30% disabled the schedule within a month, mostly due to cold mornings",
      "any": [
        "30\\s*%|30 percent|thirty percent|override|disabled|turned off|abandon|gave up"
      ]
    },
    {
      "id": "study",
      "desc": "1,150 households, cold-climate gas homes (limitation)",
      "any": [
        "1,?150|cold|midwest|natural gas|gas[- ]heated|northfield"
      ]
    }
  ],
  "allowed_numbers": [],
  "source_words": 440,
  "reference_summary": "A University of Northfield study of 1,150 cold-climate, gas-heated homes found smart thermostats cut heating bills by 8% on average (about $96 a year), far below the advertised 23%. Households that kept a schedule saved 15%; those controlling it by hand saved only 2%. About 30% disabled their schedules within a month, usually over cold mornings; starting the morning warm-up 30 minutes earlier helped."
}

What each model answered

A University of Northfield study of 1,150 upper-Midwest homes over two winters found smart thermostats cut heating bills by an average of 8% ($96/year), about a third of advertised savings. Households that kept schedules saved 15%; those controlling manually saved just 2%. Roughly 30% disabled schedules within a month, but starting morning warm-up 30 minutes earlier reduced abandonment. Break-even takes 2–3 years at average savings. Results apply mainly to cold, gas-heated homes.

{
  "facts": {
    "avg": true,
    "claim": true,
    "sched": true,
    "manual": true,
    "override": true,
    "study": true
  },
  "facts_frac": 1,
  "word_count": 72,
  "word_limit": 90,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

A study of 1,150 homes in the upper Midwest found smart thermostats save about 8% of heating bills—roughly a third of advertised savings. Households that kept schedules saved 15%, while manual users saved just 2%. About 30% disabled schedules within a month, often due to cold mornings. Setting warm-up 30 minutes earlier reduced abandonment. Break-even takes two to three years at average savings. Results may not apply to warmer areas or heat pumps.

{
  "facts": {
    "avg": true,
    "claim": true,
    "sched": true,
    "manual": true,
    "override": true,
    "study": true
  },
  "facts_frac": 1,
  "word_count": 73,
  "word_limit": 90,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

A two‑year study of 1,150 gas‑heated homes in the Upper Midwest found average heating‑bill savings of about 8 % (≈$96 per year), far less than the up to 23 % advertised by manufacturers. Savings varied widely: households that kept a programmed schedule (lowering temperature at night and when empty) saved ~15 %, while those who mostly adjusted thermostats by hand or disabled the schedule saved ~2 %. About 30 % disabled automatic settings within the first month, often because mornings felt cold. Setting the morning warm‑up 30 minutes earlier reduced schedule abandonment. Break‑even times ranged from two to three years for average savers to about one‑and‑a‑half years for high savers, depending on installation cost. Results apply mainly to cold‑climate natural‑gas homes and may not extend to heat pumps or warmer regions.

{
  "facts": {
    "avg": true,
    "claim": true,
    "sched": true,
    "manual": true,
    "override": true,
    "study": true
  },
  "facts_frac": 1,
  "word_count": 126,
  "word_limit": 90,
  "length_score": 0,
  "unsupported_numbers": [],
  "number_score": 1
}

A study of 1,150 homes found smart thermostats save less than advertised, averaging 8% (about $96/year) in heating bills, compared to the 23% often claimed. Savings vary greatly: households using schedules saved 15%, while those controlling the thermostat manually saved only 2%. Many users disabled schedules due to discomfort, especially after cold mornings. A 30-minute morning warm-up reduces abandonment, and savings take 2-3 years to break even.

{
  "facts": {
    "avg": true,
    "claim": true,
    "sched": true,
    "manual": true,
    "override": true,
    "study": true
  },
  "facts_frac": 1,
  "word_count": 67,
  "word_limit": 90,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

A two‑winter study of 1,150 U.S. homes found smart thermostats cut heating bills by only 8% on average (≈$96/year), far less than the advertised 23%. Households that kept a programmed schedule saved 15%, while those who manually overridden saved just 2%. About 30% disabled schedules within a month, often due to discomfort. Setting a 30‑minute earlier morning warm‑up reduced schedule abandonment. With device costs of $180–$300, savings of 8% break even in 2–3 years and 15% in ~1.5 years. Results apply mainly to cold‑climate, gas‑heated homes.

{
  "facts": {
    "avg": true,
    "claim": true,
    "sched": true,
    "manual": true,
    "override": true,
    "study": true
  },
  "facts_frac": 1,
  "word_count": 86,
  "word_limit": 90,
  "length_score": 1,
  "unsupported_numbers": [
    "300",
    "1.5"
  ],
  "number_score": 0
}

A large study of over 1,000 households found that smart thermostats save an average of 8% on heating bills, or about $96 a year, which is roughly a third of advertised savings. The biggest savings came from households who set up and maintained a schedule, saving 15%. However, about 30% of households quickly disabled the automatic settings, often due to discomfort. The researchers suggest setting the morning warm-up 30 minutes earlier to help households stick with their schedules.

{
  "facts": {
    "avg": true,
    "claim": true,
    "sched": true,
    "manual": false,
    "override": true,
    "study": false
  },
  "facts_frac": 0.6667,
  "word_count": 78,
  "word_limit": 90,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

A study of 1,150 homes found smart thermostats save less than advertised. On average, households cut heating bills by 8%, but those who used schedules saved 15%. Many disabled automatic settings, often due to cold mornings. The researchers suggest manufacturers default to schedules and that installation costs affect payback periods. The study's results may not apply to all climates or home types. The key takeaway: take time to set a schedule you can live with.

{
  "facts": {
    "avg": true,
    "claim": true,
    "sched": true,
    "manual": false,
    "override": true,
    "study": true
  },
  "facts_frac": 0.8333,
  "word_count": 75,
  "word_limit": 90,
  "length_score": 1,
  "unsupported_numbers": [],
  "number_score": 1
}

Smart thermostats marketed for up to 23% bill cuts, but real‑world savings average only 8% (≈$96/year) after adjusting for weather and home size. Only about one‑third of advertised savings occur; the biggest gap is between households that set schedules and those that disable them. Those who keep a schedule save 15%, while those who turn off the schedule save just 2%. A simple tweak—starting the morning warm‑up 30 minutes earlier—helps keep schedules in place. Installation costs $180–$120, so a 8% saving takes 2–3 years to break even, while a 15% saving pays off in about 1.5 years. The study’s cold‑climate, gas‑heated homes limit its relevance.

{
  "facts": {
    "avg": true,
    "claim": true,
    "sched": true,
    "manual": true,
    "override": false,
    "study": true
  },
  "facts_frac": 0.8333,
  "word_count": 105,
  "word_limit": 90,
  "length_score": 0.5,
  "unsupported_numbers": [
    "1.5"
  ],
  "number_score": 0.5
}

← summarize-extract-041All Summarize + extract itemssummarize-extract-043 →