AI Inference Guide — Agentic Design

Overview#

Core Concepts (6) · Overview · Model Advisor · Fundamentals · Non-Determinism · Interaction & APIs · Agentic Patterns · Planning & Deployment (5) · Optimization (4) · Tools & Services (2) · Operations & Practice (4)

What is AI Inference?#

AI inference is the process of using a trained ML model to make predictions or generate outputs from new input data. Unlike training, inference can be optimized for speed, efficiency, and deployment.

Edge and Device Inference#

• Privacy — Local execution reduces data sent to third parties
• Cost profile — Shift compute to user hardware
• Low latency — Avoid network round trips
• Offline capable — Work offline once models cached

Key Technologies#

• WebGPU — High-performance GPU acceleration in browsers
• WebAssembly (WASM) — Near-native CPU performance in browsers
• Model quantization — Trade model size against task quality
• ONNX Runtime — Cross-platform inference with hardware optimization