Reliability Scales Inversely: Bigger Models Compound Mistakes Faster via a Hidden Auto-Regressive Risk Regime

New research indicates that as language models increase in size, they demonstrate an inherent auto-regressive risk that causes performance reliability to degrade more rapidly despite gains in general capability. This study suggests that scaling laws may be limited by a hidden failure mechanism where larger models compound errors more aggressively than their smaller counterparts. These findings challenge the assumption that increasing model scale will consistently produce more reliable outputs, highlighting a fundamental trade-off between knowledge acquisition and systematic error propagation.