Can agents resist prompt injection, malicious documents, poisoned memories, social engineering, and compromised tools?
Prompt injection remains unsolved. Willison's 'lethal trifecta' — private data, untrusted content, and external communication in one agent — explains why agentic deployment is structurally exposed in a way chatbots weren't.
View on the map → · Open in Browse →
Prompt injection got a mechanistic account and a rare piece of good news at once. One paper traces injection to 'role confusion' — models judge text by how it sounds, not its labeled source — while a public challenge drew thousands of attempts against a hardened email agent with none succeeding, suggesting frontier training has raised the bar even though the problem is unsolved. The persistent-memory and supply-chain variants (poisoned memories, injected issue titles reaching production releases) show the attack surface widening as agents gain tools.
We trace prompt injection to role confusion: models perceive the source of text from how it sounds, not its labeled role.
Reframes injection as a role-perception failure — a 'CoT Forgery' attack imitating internal reasoning reaches ~60% success, and destyling drops it to ~10%, a mechanistic handle on both the vulnerability and a defense.
the effort the labs have been putting in to training their frontier models not to fall for injection attacks do appear effective in making these attacks much harder to pull off.
A rare piece of positive evidence: a public challenge drew ~6,000 injection attempts against an email agent with none succeeding, suggesting frontier training has meaningfully raised the bar even if injection remains unsolved.
Once written, however, this content is later retrieved as part of the agent's internal context and treated as trusted knowledge.
Addresses the poisoned-memory leg of the question: a taxonomy of six memory-poisoning attack classes, a benchmark, and evidence that existing prompt-injection defenses fail to protect persistent agent memory (avg attack success ~50%).
an attacker can craft an issue title containing instructions that trick Claude into running arbitrary commands.
A real supply-chain compromise: a prompt injection in a GitHub issue title chained through cache poisoning to steal publishing secrets and push a malicious release — indirect injection reaching production software distribution.
The biggest challenge in designing agentic systems continues to be preventing them from enabling attackers to exfiltrate data.
Injection plus agent-sent emails plus pre-authenticated links produced a working file-exfiltration chain in a shipped agent — a full lethal-trifecta instantiation.