How stories are grouped and scored
Every ranking on this site comes from a written-down process, not an editor's mood and not engagement metrics. This page explains that process in plain language, including the parts where AI models are involved and the limits of what the score means.
How stories are grouped
Model Beat watches roughly 45 feeds (lab blogs, tech press, arXiv, Hacker News, and Google News queries that reach 1,300+ outlets) every 3 hours. When several outlets cover the same event, those articles are grouped into one story, so the feed shows events rather than fifteen copies of the same headline.
Grouping compares the meaning of headlines and excerpts using text embeddings, with a deliberately strict similarity bar: two different stories shown as one would mislead you, while one story split in two is merely quieter. Because short headlines about the same event sometimes phrase it too differently for embeddings alone, borderline pairs get a second check: a small language model is asked whether two headlines describe the same event, and only a clear yes merges them. If that check fails or is unavailable, nothing merges. The system's mistakes are built to lean toward splitting, never toward false merging.
How the significance score is computed
The 1-10 score on every story combines three signals:
- Who is covering it. Each outlet carries an authority weight: a top lab's own announcement or a major wire service counts for more than an aggregator or a syndicated local affiliate. Weights apply per organization, so two blogs owned by the same company count once.
- How many independent organizations corroborate it. More distinct, credible newsrooms raise the score, with diminishing returns and weighted by quality: sixteen syndicated copies of one wire story cannot outrank original reporting from two major outlets.
- How consequential the content is. An LLM rates each article's impact on a 1-10 rubric: routine updates land 3-4, notable launches 5-6, major releases from top labs 7-8, and 9-10 is reserved for paradigm-shifting events. We disclose this openly: an LLM rates impact, and its ratings are bounded so one bad grade cannot dominate a well-corroborated story.
The three signals multiply together and are compressed onto the 1-10 display scale so the top stays meaningful: a single announcement from a top lab lands around 5, a story corroborated across many major outlets lands around 8, and 10 is reserved for the handful of stories a year that everyone covers. Click any score badge to see that story's own inputs: how many articles, how many sources, and the impact rating.
What the score is not
- Not a truth score. It measures the weight and breadth of coverage plus rated impact, not whether claims in the coverage are correct.
- Not personalized. A story that matters enormously to your stack may score 4 because few outlets covered it. That is what the model tracker and its price-change records are for.
- Not engagement-based. Clicks, likes, and shares play no part, which is a deliberate anti-hype choice.
Sources and citations
Every story card names its sources, links directly to the original articles, and shows timestamps. Excerpts stay short, and summaries are original text written per story, never copied phrasing. Model data comes from Epoch AI (benchmarks, CC-BY) and OpenRouter (pricing and specs), credited on every model page.
Think a story was scored wrong? Tell me. The scoring rules have changed before in response to measured mistakes, and that is by design.