AI Technology RadarAI Technology Radar

Prompt Injection Awareness

securityprompting
Adopt

Update 07/2026: Prompt injection remains unsolved - but the defense thinking has matured beyond the filter-based mitigations in our previous entry. Input validation and guardrails are still important but alone are now considered insufficient for critical guards. You need architectural defense, enforced by the harness - not by the model.

The "Lethal Trifecta"

The now-standard mental model for agent exfiltration risk. An agent is exfiltration-prone by construction when it combines all three:

  1. Access to private data - like sensitive files, databases, internal APIs, user context
  2. Exposure to untrusted content - web pages, incoming mails/issues/tickets, third-party tool results
  3. An external communication channel - HTTP requests, mail sending, writing to public locations

Design rule: break at least one leg per agent. Examples:

  • A research agent that reads the web should not hold private credentials.
  • An agent with database access should not be able to call arbitrary URLs.

Architectural defense patterns

  • Plan-then-execute: the agent fixes its plan before reading untrusted content - later content may influence parameters, but cannot add new actions.
  • Dual-LLM / quarantined model: a model without any tool access processes untrusted content and returns only structured, constrained results to the privileged agent.
  • Capability policies in the harness:
    • least-privilege tool access, allow/ask/deny rules
    • read-only defaults for tools that consume external content
    • separate agents/contexts per trust level instead of one agent with all permissions
  • Human-in-the-loop approvals for critical actions (irreversible operations, external communication, credential use) - see the approval section in Harness Engineering. Beware approval fatigue: reserve approvals for genuinely critical actions, or users will click through them.

New attack surfaces since our last entry

  • MCP-based injection: tool descriptions and tool results of third-party MCP servers are untrusted input; a malicious or compromised server can steer the agent. Restrict allowed servers and review what they expose.
  • Web content targeting agents: malicious injections placed in public web pages increased measurably since late 2025 - browsing agents are now an explicit target group.
  • Memory poisoning: injected content that persists into agent memory silently compromises future sessions (see ASI06 in the OWASP Agentic Top 10).

Official guidance & testing

Adopt

Prompt injection is a critical security vulnerability where attackers manipulate LLM inputs to bypass security controls, leak data, or alter model behavior. It ranks as #1 in the OWASP LLM Top 10.

Attack Types

  • Direct Injection: Malicious prompts overriding system instructions
  • Indirect Injection: Attacks through contaminated data sources
  • Context Manipulation: Exploiting model's context handling
  • Chain Attacks: Multi-step manipulation sequences

Key Risks

  • System prompt exposure
  • Sensitive data leakage
  • Security control bypass
  • Unauthorized actions
  • Output manipulation

Mitigations

  • Input validation and sanitization
  • Guardrails
  • Output filtering and monitoring
  • Permission boundaries
  • Regular security testing with Garak
  • Comprehensive logging

Resources

Related Topics