LLM Gateway

LLM Gateway sits between your applications and the model providers they call, so adding or swapping a provider stops being an application change. It routes to OpenAI, Anthropic, Google Vertex AI and others, holds every provider key in one place, and records what each call consumed and cost.

Because the gateway speaks the OpenAI API format, existing clients migrate by pointing at a different base address rather than rewriting request code. The repository is a monorepo holding a Next.js dashboard, a Hono backend, the gateway service itself, a documentation site, a Drizzle ORM schema with migrations, and shared model and provider definitions.

The parts a team actually uses break down as follows.

• Unified interface: one request and response shape works across every connected provider.
• Key management: provider credentials live in a single place instead of scattered through services.
• Usage analytics: requests, tokens, response times, and costs are tracked across all interactions.
• Performance monitoring: models are compared on speed and cost-effectiveness before you commit to one.
• Self-hosting: a unified Docker image bundles Postgres and Redis and starts from two generated secrets.

Self-hosters should use Docker-managed volumes rather than bind-mounting a host directory to the Postgres data path, since initialization inside the container needs to set permissions there and can fail depending on host filesystem and ownership. A hosted account is available for teams who want an API key immediately. The core is AGPLv3, while commercial features in the enterprise directory need a separate license.