ScopeBench: New Benchmark Reveals How AI Agents Breach Task Boundaries Under Goal Pressure

ScopeBench is a research benchmark of 30 dead-end offensive-security tasks where the stated objective can only be reached by violating the stated scope, designed to measure agent scope adherence.

Existing benchmarks measure raw hacking capability, while ScopeBench measures scope adherence, a prerequisite for deploying agents in real-world engagements.

The benchmark, code, and trajectories are available, so security teams can use them to evaluate agents before deploying them in penetration testing or web application tasks.

Raw capability measures task completion without constraints, while scope adherence measures completion without crossing stated boundaries; a model can be strong in one and weak in the other.