Vulnerability / research

Planning before browsing resisted prompt injection in a web-agent benchmark

Reported

Published here

What happened

Researchers compared Baya, which commits to permitted website operations before seeing live content, with agents that choose their next action while browsing. In the WASP tests, malicious page content redirected some reactive agents but did not redirect Baya.

Parties and reporting

AI tools and systems: Baya web-agent prototype, ReAct web-agent baselines, WASP security benchmark

Outcome

The authors report 0% attack success for Baya, versus 16.2% for the reactive agent with API access and 25.5% for the browser-only agent. These are controlled benchmark results with prepared interfaces, not evidence that arbitrary websites are safe.

Operating lesson

Decide which actions a web agent may take before it encounters untrusted content. Test both unauthorized actions and wrong results within an authorized action; those are different failure modes.

Evidence and limits

The specific Baya evaluation was submitted November 4 and appeared in arXiv’s November 5 public listing. An earlier position paper discussed the architecture. The evaluation concerns control over actions, not false website data, service outages, or compromise of the trusted website interface.

Sources