Portkey AI Gateway

The AI Gateway routes requests to more than sixteen hundred language, vision, audio and image models behind one API, so changing provider stops being a code change. It is small, around 122kb, adds under a millisecond of latency, and the project reports more than ten billion tokens passing through it every day.

Running it locally is a single npx command, after which the gateway serves on a local port and a console shows every request that went through. Calls can come from the Portkey client, the OpenAI SDKs, or framework integrations for LangChain, LlamaIndex, Autogen and CrewAI, all using the familiar OpenAI signature.

Behavior is set with configs, small objects attached to a client.

• Reliable routing: fallbacks to another provider or model on chosen errors, up to five automatic retries with exponential backoff, weighted load balancing and granular timeouts.
• Guardrails: more than forty prebuilt checks on inputs and outputs, with room to bring your own or use partner checks.
• Cost management: simple and semantic caching, plus analytics covering request volume, latency, cost and error rates.
• MCP Gateway: one authentication layer, access control and full logging for Model Context Protocol servers across an organization.

Deployment guides cover Docker, Node.js, Cloudflare Workers and Replit as well as the hosted service, while the enterprise build adds organization management, governance and private deployments on AWS, Azure, GCP, OpenShift and Kubernetes. Gateway 2.0, which merges the enterprise core into the open source project, is available as a pre-release branch.