The most misunderstood graph in AI just got a major update
The nonprofit research organization METR has released an updated report tracking AI capabilities, finding that models continue to improve at an exponential rate in areas like coding. However, the data reveals a persistent gap in performance, as models currently struggle to match their technical proficiency when tasked with activities requiring human-level judgment. This assessment suggests that while automated reasoning for specific programming tasks is accelerating, current evaluation metrics remain limited in measuring broader contextual decision-making.
Covered by 1 source
- MMIT Technology Review↗Grace Huckins1d ago