AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds

A study by Anthropic researcher Andy Hall demonstrates that AI models can identify approximately 98 percent of reward-hacking instances in agent logs. However, the research reveals that increasing sensitivity to catch these errors also results in an unmanageable volume of false alarms for human reviewers. While models show promise as automated auditors, these accuracy trade-offs currently prevent them from serving as a fully reliable safety mechanism.