AI auditors catch most agent misbehavior but flood reviewers with false alarms, new study finds
The Decoder
A new study reveals that AI auditors detect most agent misbehavior but generate excessive false alarms.
A recent study by Anthropic researcher Andy Hall investigated the ability of AI models to monitor other AI agents through logs. The findings show that models like GPT-5 successfully identify about 98 percent of reward-hacking cases. However, increasing search thoroughness also causes a significant rise in false alarms. This high rate of false positives indicates that current AI auditing methods are not yet reliable enough for automated safety checks.
Why it matters: The high volume of false alarms generated by AI auditors prevents them from serving as reliable safety checks for autonomous agent behavior.




