Arize Phoenix

Arize Phoenix is an open source AI observability platform built for experimentation, evaluation, and troubleshooting. It removes the guesswork from LLM applications, where a bad answer can come from a prompt, a retrieval step, or a model call, and nothing in an ordinary log tells you which.

Instrumentation is OpenTelemetry based through the OpenInference project, which keeps Phoenix vendor and language agnostic. It ships with support for frameworks such as LangGraph, CrewAI, LlamaIndex, DSPy and the OpenAI Agents SDK, and for providers including OpenAI, Anthropic, Google GenAI, AWS Bedrock and LiteLLM.

The platform is organized around a handful of workflows.

• Tracing: capture your application runtime and inspect every span sitting behind an answer.
• Evaluation: use LLMs to benchmark response and retrieval quality against criteria you define.
• Datasets and experiments: version examples, then track how prompt, model and retrieval changes score.
• Playground: compare models, adjust parameters, and replay traced calls while tuning a prompt.
• Prompt management: version, tag and test prompt changes instead of editing them in place.

A remote MCP server exposes traces, datasets and experiments to clients such as Claude Code and Cursor, and a setup command in the CLI wires instrumentation into an existing project by detecting its framework and provider.

Phoenix installs with pip or conda and runs locally, in a container, or on a cluster through published Docker images and a Helm chart. It is self hosted throughout, so traces stay inside your own infrastructure.