All posts
Research
·
11 min read
A field guide to agent failure modes
We analyzed 1.2 million failed agent runs. Almost all of them fall into seven patterns — and most are preventable with a single eval.
Iris Chen
Research Lead
Failures cluster, and that’s good news
When we started labeling failed runs across Sable customers, we expected a long tail of one-off bugs. Instead, 91% of failures fell into seven recurring patterns: tool misuse, stale retrieval, plan loops, premature answers, instruction drift, unsafe actions and silent truncation.
Recurring patterns are good news. They mean a small set of targeted evals can catch the majority of problems before customers do.
The three that matter most
Tool misuse: valid-looking calls with wrong arguments
Plan loops: agents retrying the same step with no new information
Premature answers: responding before enough context was gathered
We’ve published scorers for all seven patterns in the Sable eval library, free on every plan.