# Multimodal Attacks


📂 ai-red-teaming

## AI RED TEAMING TECHNIQUES

# Multimodal Attacks

Cross-modal exploitation and modality-specific attack techniques

## Available Techniques

### Image-Based Prompt Injection

Embedding malicious text instructions or prompts within images to bypass text-based content filters and inject harmful directives through the visual modality.

#### KEY FEATURES

- •
Hidden text in images

- •
Visual prompt injection

- •
OCR exploitation

#### PRIMARY DEFENSES

- •
Image content analysis

- •
OCR output sanitization

- •
Cross-modal validation

#### KEY RISKS

### Cross-Modal Confusion Attack

Exploiting inconsistencies or conflicts between different input modalities to confuse the AI system and bypass security controls or trigger unintended behaviors.

#### KEY FEATURES

- •
Modality conflict exploitation

- •
Contradictory input injection

- •
Priority manipulation

#### PRIMARY DEFENSES

- •
Cross-modal consistency validation

- •
Modality agreement requirements

- •
Conflict detection and rejection

#### KEY RISKS

### Audio Adversarial Examples

Crafting audio inputs with imperceptible perturbations that cause speech recognition or audio processing systems to misinterpret commands or bypass security measures.

#### KEY FEATURES

- •
Imperceptible audio perturbations

- •
Speech recognition manipulation

- •
Command misinterpretation

#### PRIMARY DEFENSES

- •
Audio perturbation detection

- •
Speech pattern validation

- •
Multi-model audio verification

#### KEY RISKS

### Video Manipulation & Injection

Manipulation of video streams or recorded content to inject malicious visual sequences, subliminal frames, or adversarial patterns that compromise video understanding systems.

#### KEY FEATURES

- •
Frame injection

- •
Subliminal content insertion

- •
Temporal attack patterns

#### PRIMARY DEFENSES

- •
Frame-by-frame validation

- •
Temporal consistency checks

- •
Subliminal content detection

#### KEY RISKS

### Sensor Data Poisoning

Manipulation of sensor inputs (IoT devices, environmental sensors, biometric readers) to feed false data to AI systems and compromise decision-making in autonomous systems.

#### KEY FEATURES

- •
Sensor input manipulation

- •
Environmental data falsification

- •
Biometric spoofing

#### PRIMARY DEFENSES

- •
Sensor data validation

- •
Multi-sensor verification

- •
Anomaly detection algorithms

#### KEY RISKS

### Modality-Specific Jailbreaking

Bypassing content filters and safety measures by exploiting weaknesses in specific modality processing, using less-protected input channels to circumvent text-based safeguards.

#### KEY FEATURES

- •
Modality-specific filter bypass

- •
Weak channel exploitation

- •
Alternative input abuse

#### PRIMARY DEFENSES

- •
Unified safety filters across modalities

- •
Equivalent protection per channel

- •
Cross-modal content analysis

#### KEY RISKS

### Embedding Space Manipulation

Crafting inputs across multiple modalities that occupy similar positions in embedding space to confuse similarity matching, retrieval, or classification systems.

#### KEY FEATURES

- •
Embedding collision creation

- •
Similarity exploitation

- •
Retrieval manipulation

#### PRIMARY DEFENSES

- •
Embedding space validation

- •
Multi-modal consistency checking

- •
Semantic verification

#### KEY RISKS

### Cross-Modal Transfer Attack

Crafting adversarial examples in one modality that successfully transfer to compromise other modalities, exploiting shared representations in multimodal models.

#### KEY FEATURES

- •
Transferability exploitation

- •
Shared representation attacks

- •
Cross-modal perturbations

#### PRIMARY DEFENSES

- •
Modality-specific processing

- •
Transfer detection mechanisms

- •
Independent validation per modality

#### KEY RISKS

### Multimodal Backdoor Attack

Inserting backdoors that activate only when specific combinations of inputs across multiple modalities are present, creating stealthy trigger-based compromises.

#### KEY FEATURES

- •
Multi-modal trigger conditions

- •
Combination-based activation

- •
Stealthy backdoor insertion

#### PRIMARY DEFENSES

- •
Training data validation

- •
Backdoor detection algorithms

- •
Multi-modal integrity checks

#### KEY RISKS

### Modality Prioritization Exploitation

Exploiting the system's prioritization or weighting of different input modalities to bypass security controls by manipulating lower-priority channels.

#### KEY FEATURES

- •
Priority order exploitation

- •
Weight manipulation

- •
Low-priority channel abuse

#### PRIMARY DEFENSES

- •
Balanced modality processing

- •
Equal validation across channels

- •
Dynamic priority adjustment

#### KEY RISKS

### Ethical Guidelines for Multimodal Attacks

When working with multimodal attacks techniques, always follow these ethical guidelines:

- • Only test on systems you own or have explicit written permission to test

- • Focus on building better defenses, not conducting attacks

- • Follow responsible disclosure practices for any vulnerabilities found

- • Document and report findings to improve security for everyone

- • Consider the potential impact on users and society

- • Ensure compliance with all applicable laws and regulations

FROM THE ENGINEER BEHIND THIS CATALOG

## Get your agent system red-teamed

The attacks documented here work on production agent systems every day. Have yours tested before someone else does: prompt injection, jailbreaks, tool misuse and data exfiltration, with every finding written up next to its fix.

€750 instead of €1,500, one week, written report and walkthrough call, until 30 September

## AI Red Teaming

## AI Red Teaming Techniques