# Prompt Injection


📂 ai-red-teaming

## AI RED TEAMING TECHNIQUES

# Prompt Injection

Techniques to manipulate AI responses through malicious prompts

## Available Techniques

### Basic Prompt Injection

Fundamental techniques to inject malicious instructions into AI prompts to bypass intended behavior.

#### KEY FEATURES

- •
Simple instruction override

- •
Context manipulation

- •
Role confusion attacks

#### PRIMARY DEFENSES

- •
Input sanitization and validation

- •
Prompt template isolation

- •
Context boundaries enforcement

#### KEY RISKS

### Indirect Prompt Injection

Advanced technique where malicious instructions are embedded in external content that the AI processes.

#### KEY FEATURES

- •
Hidden instruction embedding

- •
Content-based manipulation

- •
Cross-context attacks

#### PRIMARY DEFENSES

- •
Content preprocessing and sanitization

- •
Source validation and verification

- •
Context isolation mechanisms

#### KEY RISKS

### Many-Shot Jailbreaking

Advanced technique using large number of harmful question-answer pairs to gradually shift model behavior through in-context learning.

#### KEY FEATURES

- •
In-context learning exploitation

- •
Gradual behavior modification

- •
128+ shot examples

#### PRIMARY DEFENSES

- •
Context window limitations

- •
Few-shot example filtering

- •
Constitutional AI training

#### KEY RISKS

### Indirect Prompt Injection via External Content

Embedding malicious instructions in external content that AI systems process, causing unintended behaviors when the content is ingested.

#### KEY FEATURES

- •
Hidden instruction embedding

- •
Cross-system contamination

- •
Persistent attack vectors

#### PRIMARY DEFENSES

- •
Content preprocessing and sanitization

- •
Instruction filtering from external sources

- •
Context isolation between user and external content

#### KEY RISKS

### Copy-Paste Injection Attack

Embedding hidden malicious prompts in copyable text that execute when pasted into AI systems, exploiting user trust in copied content.

#### KEY FEATURES

- •
Hidden instruction embedding

- •
Clipboard exploitation

- •
User behavior manipulation

#### PRIMARY DEFENSES

- •
Unicode normalization and filtering

- •
Character set validation

- •
Hidden content detection

#### KEY RISKS

### System Prompt Leakage Attacks

Techniques to extract hidden system prompts, instructions, and configuration details from AI systems.

#### KEY FEATURES

- •
System instruction extraction

- •
Configuration revelation

- •
Hidden prompt discovery

#### PRIMARY DEFENSES

- •
System prompt isolation techniques

- •
Instruction filtering and detection

- •
Response content filtering

#### KEY RISKS

### Policy Puppetry Configuration Attack

Formatting prompts as configuration files (XML, JSON, INI) to bypass content policies by disguising harmful requests as system configurations.

#### KEY FEATURES

- •
Configuration file mimicry

- •
Policy circumvention

- •
Format-based deception

#### PRIMARY DEFENSES

- •
Configuration format detection and blocking

- •
Structured input validation

- •
Content-agnostic policy enforcement

#### KEY RISKS

### ASCII Art Injection Attack

Using ASCII art and visual text manipulation to bypass AI content filters that may not properly parse visual or artistic text representations.

#### KEY FEATURES

- •
Visual obfuscation techniques

- •
ASCII art exploitation

- •
Character pattern manipulation

#### PRIMARY DEFENSES

- •
ASCII art pattern recognition

- •
Character sequence normalization

- •
Visual text parsing and analysis

#### KEY RISKS

### Ethical Guidelines for Prompt Injection

When working with prompt injection techniques, always follow these ethical guidelines:

- • Only test on systems you own or have explicit written permission to test

- • Focus on building better defenses, not conducting attacks

- • Follow responsible disclosure practices for any vulnerabilities found

- • Document and report findings to improve security for everyone

- • Consider the potential impact on users and society

- • Ensure compliance with all applicable laws and regulations

FROM THE ENGINEER BEHIND THIS CATALOG

## Get your agent system red-teamed

The attacks documented here work on production agent systems every day. Have yours tested before someone else does: prompt injection, jailbreaks, tool misuse and data exfiltration, with every finding written up next to its fix.

€750 instead of €1,500, one week, written report and walkthrough call, until 30 September

## AI Red Teaming

## AI Red Teaming Techniques