Langfuse

Langfuse is an open source LLM engineering platform for teams that develop, monitor, evaluate and debug AI applications together. It replaces the improvised mix of log files and spreadsheets most projects fall back on when they need to work out why a particular model call went wrong.

Instrumentation sends traces of LLM calls and the logic around them, retrieval, embedding and agent actions, so a complex run or a whole user session can be inspected step by step. Prompt management is centralized with version control and collaborative iteration, and caching on both server and client means changing a prompt adds no latency to the running application.

Four further pieces close the development loop.

• Evaluations: LLM-as-a-judge, code evaluators, user feedback, manual labeling and custom pipelines through the API.
• Datasets: test sets and benchmarks for pre-deployment testing and structured experiments.
• Playground: prompts and model configurations are iterated directly, reached from a bad result seen in a trace.
• API access: an OpenAPI spec, a Postman collection and typed Python and JavaScript SDKs power bespoke workflows.

Integrations cover the two SDKs, a drop-in replacement for the OpenAI SDK, callback handlers for LangChain, LlamaIndex and Haystack, and LiteLLM for reaching a hundred or more models. Self-hosting takes about five minutes locally with Docker Compose, runs on a single virtual machine, or goes to Kubernetes with Helm as the preferred production route. A managed cloud with a free tier exists for teams who would rather skip all of that.