OpenLLMetry

OpenLLMetry extends OpenTelemetry to cover LLM applications, so model calls, vector database queries and agent frameworks appear in the observability stack a team already runs. Nothing has to move to a separate vendor dashboard built only for AI.

The repository holds standard OpenTelemetry instrumentations for LLM providers and vector databases, plus a Traceloop SDK that shortens the setup. Getting started is a pip install and a single init call in your code. Teams already instrumented with OpenTelemetry can skip the SDK and add the instrumentations directly.

Coverage spans the layers an LLM request passes through.

• Model providers: instrument OpenAI, Anthropic, Bedrock, Vertex AI, Cohere, Mistral, Groq, Ollama, HuggingFace and more.
• Vector databases: cover Chroma, Pinecone, Qdrant, Weaviate, Milvus, LanceDB and Marqo.
• Frameworks: trace LangChain, LlamaIndex, LangGraph, CrewAI, Haystack, LiteLLM and OpenAI Agents.
• Protocol support: instruments the Model Context Protocol alongside the frameworks that use it.
• Destinations: export to Datadog, Honeycomb, Grafana, New Relic, Splunk, Sentry, SigNoz, Dynatrace and around two dozen more.

Everything OpenTelemetry already instruments, a database or an outbound API call, keeps working next to these extensions. Traceloop maintains the project under the Apache 2.0 license and has removed telemetry collection from both the SDK and the instrumentations. The semantic conventions it introduced are now part of OpenTelemetry itself, and a JavaScript and TypeScript version lives in a sibling repository.