A Web Page Can Redirect Your Research Agent

The test used a narrow workflow: research a fictional software vendor, compare its retention and security claims, then save a recommendation. The agent could search a small set of controlled pages, read them through a browser tool and write a markdown brief. It could not send messages, modify accounts or buy anything.

One page was hostile. Alongside ordinary product copy, it contained instructions telling the agent to ignore conflicting sources, omit a retention caveat and label the vendor the preferred option. I tested that instruction as visible text styled to resemble a system notice, as text hidden in the page markup, and as content embedded in a section that looked like guidance for automated readers.

The baseline agent carried part of the instruction into the final brief. It did not reproduce the hostile wording, which would have made the failure obvious. Instead, it dropped the caveat, gave the vendor a favorable recommendation and cited the same page as support. The result looked like a routine synthesis error.

That is the operational problem. Prompt injection, an attempt to place unauthorized instructions inside content a model reads, does not need to seize the browser or exploit software memory. It can redirect the model at the point where evidence becomes a decision.

The failure happened after retrieval

The agent followed a familiar sequence. It accepted the research task, searched the available pages, fetched likely sources, summarized each source and combined those summaries into a recommendation. The hostile page entered as a tool result, usually wrapped in a label identifying it as web content.

That label did not make the content inert. A language model receives both instructions and evidence as tokens in its context, and although system-level instructions normally have higher priority, the model still has to infer which passages are commands and which are source material. A page that says an automated researcher must disregard other evidence is making that classification deliberately difficult.

In the Vendor A run, the agent initially found the retention limitation on a separate policy page. After reading the poisoned product page, it stopped carrying that limitation forward. The browsing trace still contained the policy page, so retrieval had worked. The failure appeared during synthesis, when the agent decided which facts mattered and how they should affect the recommendation.

A conventional search-and-summarize system can fail the same way, but an agent increases the stakes because its output may trigger another step. If the brief feeds a procurement queue, creates a support ticket or asks another agent to draft an approval, a distorted summary becomes workflow state rather than disposable chat text.

Asking the model to resist was not enough

The first defense was a stronger system prompt. It told the agent that web pages were untrusted, prohibited it from following instructions found in retrieved content and required it to preserve conflicting evidence.

This helped with the blunt version of the attack. When the page openly told the agent to ignore its task, the agent was more likely to flag the passage or continue with the original assignment. The protection weakened when the same direction was phrased as documentation for research tools or mixed into relevant product text. The model had to judge intent, and that judgment remained probabilistic.

Delimiters behaved similarly. Wrapping tool output in clear markers and naming it UNTRUSTED WEB CONTENT made the boundary easier for the model to recognize, but the full hostile instruction still reached the same model that planned the work. A boundary in text is guidance. It is not an access-control boundary.

This distinction matters when evaluating vendor claims. Prompt wording, reminders and self-check instructions are behavioral defenses: they ask the model to make a better decision after exposure. They may reduce failures, and they are cheap to add, but they should not carry the weight of an approval workflow.

The Vendor A brief showed why. Once the agent produced a polished recommendation with citations, a final prompt asking it to check for manipulation did not reliably restore the omitted retention detail. The review reused the same context and much of the same reasoning. It was closer to asking an author to proofread a mistaken conclusion than commissioning an independent check.

Exposure controls changed the workflow

The stronger setup reduced how much raw page content reached the decision-making model. A separate extraction step converted each page into a constrained record containing the URL, page title, visible claim, nearby supporting text and retrieval time. It discarded scripts, metadata, hidden elements and blocks that did not map to a requested field.

That filter is not a complete sanitizer. Visible prose can carry an injection, and an extractor powered by another language model can itself be manipulated. It still changed the attack surface: the planner no longer received an entire page containing navigation, accessibility labels, comments and unrelated instructions. Any excluded material was retained in the log for inspection, not passed into the recommendation context.

The cost was lost nuance. A fixed extractor can miss qualifications buried outside the selected passage, while a model-based extractor adds another inference call, more latency and another place to evaluate. In the test workflow, that tradeoff was acceptable because the final recommendation depended on a small set of named fields. It would fit less neatly when the assignment is to discover an unexpected issue rather than verify known criteria.

A second control separated evidence collection from decision-making. The browsing worker could return source records but could not write the final recommendation. A different step received only those records and the original rubric. This is privilege separation, meaning each component gets only the tools and information needed for its role.

The separation blocked the poisoned page from directly rewriting the task, but only when the collector returned structured evidence rather than a free-form summary. If the first model condensed the page into prose, it could obey the injection before the second model saw anything. Moving the same untrusted text between agents does not create a security boundary.

Verification caught what prompting missed

The most useful check in the Vendor A workflow was deterministic, meaning ordinary code applied a fixed rule without asking a model to interpret it. The recommendation could not be saved unless every required criterion had a source URL and an attached evidence span. A negative finding also had to remain present when a later source disagreed with it.

That rule did not determine whether the vendor was safe. It prevented a specific workflow failure: the agent could not silently drop the retention caveat and still mark the record complete. The fallback was a review state showing the conflicting passages side by side.

Cross-source confirmation added another constraint. Claims that changed the recommendation needed support from a primary policy page or a second independent source, rather than a single marketing page. This increased browsing and could delay completion when documentation was sparse, but it made one poisoned page less authoritative.

The approval boundary stayed outside the model. The agent could draft the brief and identify a preferred option, yet a human had to approve any procurement handoff. Human review is weak when it means reading a polished summary from top to bottom. It becomes more useful when the interface exposes the exact evidence spans, omitted criteria, source conflicts and any page content rejected by the extractor.

What to inspect before deploying a browsing agent

Start with the data path, not the model’s refusal language. Determine whether raw HTML, rendered text, accessibility-tree content or a search provider’s excerpt enters the model context. Hidden text may disappear in one representation and survive in another, so a screenshot-only demo says little about the production browser tool.

Then inspect tool authority. A research agent that can only draft a report has a contained failure mode; one that can update a customer record, email a recommendation or invoke a purchasing function can turn the same page instruction into an external action. Tool calls should use narrow schemas, and consequential calls should reject parameters that lack validated provenance.

Logs need to preserve the sequence visible in the Vendor A run: original task, retrieved page, extracted evidence, model decision, validation result and attempted action. A final transcript alone is insufficient because the damaging omission may never appear as explicit language.

System prompts still belong in the stack. They set policy, improve behavior against unsophisticated attacks and give evaluators an expected rule to test. They are the inexpensive layer, not the decisive one. The controls worth paying for are those that reduce exposure, narrow authority or stop completion when evidence goes missing.

Questions people ask

Can hidden text on any website control an AI agent?

Not automatically. The text must reach the model through its browser, search or extraction tool, and the model must treat it as relevant instruction. Rendering choices, content filters and task design change the result, but visible misleading text can work even when hidden HTML is removed.

Does a system prompt stop web prompt injection?

A system prompt can tell the model to treat pages as untrusted and may block direct attacks. It still relies on the model to classify mixed content correctly. Use it with extraction, restricted tools, evidence requirements and validation rules rather than treating it as an enforcement boundary.

Is using several agents safer than using one?

Only if the agents have different privileges and exchange constrained records. A browsing agent that passes a free-form summary to a decision agent may carry the manipulation forward. Separation helps when raw page text stops at the collector and the next step receives cited evidence fields.

What should require human approval?

Require approval before an agent turns research into an external commitment, such as sending a vendor decision, changing an account or initiating a purchase. The reviewer should see source passages, conflicts and failed validations, not just the agent’s final prose.