LiteLLM Alternatives: Open Source Gateways Compared

Five open source gateways can realistically replace LiteLLM Proxy today: Bifrost, Portkey AI Gateway, Kong Gateway's AI plugins, MLflow AI Gateway, and Helicone, each with a different license, footprint, and standout feature. LiteLLM itself stays open source under an MIT core, but the operational load of running it (a Postgres database, a Redis cache, and an enterprise license once you need SSO or audit logs) is what pushes self-hosters to look elsewhere. This guide compares the real, verifiable tradeoffs: license, self-host footprint, and standout feature, not a vendor's ranked list of itself first.

Key takeaways:

  • LiteLLM Proxy's OSS core is MIT licensed, but SSO past 5 users, RBAC, audit logs, and several moderation callbacks sit behind a separate enterprise license (Source: LiteLLM Docs).
  • The credible LiteLLM alternatives that stay open source are Bifrost (Go, Apache 2.0), Portkey AI Gateway (TypeScript, MIT), Kong Gateway's AI plugins (Lua core, Apache 2.0), MLflow AI Gateway (Python, Apache 2.0), and Helicone (TypeScript, Apache 2.0).
  • Vendor benchmarks (TrueFoundry's latency numbers, Bifrost's "50x faster than LiteLLM" claim) are self-reported by the company selling the product; treat them as directional, not independently verified.
  • Self-host footprint varies more than feature lists suggest: LiteLLM needs Postgres plus Redis at 2+ replicas, while Bifrost and Portkey can start from a single binary or npx command.

Why self-hosters look for a LiteLLM alternative

LiteLLM Proxy is the default choice for teams routing agent traffic through one OpenAI-compatible endpoint, and its GitHub repository shows why: 54.9k stars and support for 100+ providers is a hard base to beat (Source: LiteLLM Docs). The reasons people search for an alternative are almost always operational, not feature-driven. Running LiteLLM at scale means a PostgreSQL database for keys, teams, and spend logs, plus Redis once you run more than one instance, for rate limiting and router state (Source: LiteLLM Docs).

A competitive post from TrueFoundry, published July 1, 2026, put a number on the pain point: it claims its own gateway delivers roughly 3 to 4 ms of latency at 350+ requests per second on a single vCPU, against what it calls LiteLLM's "significant latency." That number comes from TrueFoundry's own blog, ranking its own product first with a chart and no published methodology, so read it as a competitor's marketing claim, not a neutral benchmark. The underlying operational complaint, though, tracks with what LiteLLM's own docs describe: a stateful proxy with a database and a cache is simply more to run than a stateless binary.

What "open source" should mean here

Is LiteLLM open source? Yes, the core proxy is MIT licensed and covers the gateway, virtual keys, spend tracking, budgets, fallbacks, and logging. But the enterprise/ directory in the same repository carries a separate license, and features like single sign-on beyond five users, role-based access control, audit logs, and several moderation callbacks require that enterprise license (Source: LiteLLM Docs). That split matters when you are picking a "true" open source alternative: a gateway that gates its access-control layer behind a paid tier is not fully open, even if the core is.

Each alternative below is judged on the same axis: what license actually covers the self-hosted binary you would run, not just the marketing page's "open source" badge.

Five LiteLLM alternatives that stay open source

Bifrost: the fastest self-reported option, made by an eval vendor

Bifrost is a Go-based gateway released under Apache 2.0, with 23+ supported providers including OpenAI, Anthropic, AWS Bedrock, and Google Vertex (Source: Bifrost GitHub). Its README claims 11 microseconds of added latency on a t3.xlarge instance and a 100% success rate at 5,000 requests per second, framed as "50x faster than LiteLLM." Those are the maker's own numbers, not an independent test, and worth verifying against your own traffic before trusting them outright. One disclosure the SERP rarely makes clear: Bifrost is built and maintained by Maxim AI, the same company whose blog post about Bifrost also ranks on this exact query, so the "neutral review" framing some pages use is not neutral.

Portkey AI Gateway: the lightest footprint

Portkey's open source gateway is MIT licensed, written in TypeScript, and starts with a single npx @portkey-ai/gateway command with no separate database required for the base gateway (Source: Portkey AI Gateway GitHub). It advertises a "122kb" package size and sub-millisecond latency, plus 250+ models across 45+ providers, automatic retries, and an MCP gateway for Model Context Protocol servers. Portkey the company also runs a hosted cloud version of the same gateway. Inference: since the MIT-licensed repository is what you would actually self-host, confirm you are pointing at that repository rather than the managed cloud endpoint before counting it as fully self-hosted.

Kong Gateway's AI plugins: the choice if you already run Kong

Kong Gateway is Apache 2.0 and one of the most widely deployed API gateways, with 43.9k GitHub stars; its AI Gateway plugins route to OpenAI, Anthropic, Bedrock, Vertex, Azure, and Mistral on top of the same gateway teams already use for regular API traffic (Source: Kong Gateway GitHub). It supports a database-backed mode with PostgreSQL or a declarative, database-less mode for lighter deployments. The standout feature is not LLM routing alone; it is folding LLM traffic into infrastructure you may already operate, rather than standing up a new proxy.

MLflow AI Gateway: the pick for teams already on MLflow

MLflow AI Gateway is Apache 2.0, backed by the Linux Foundation, and runs as part of the existing MLflow Tracking Server rather than a separate service (Source: MLflow AI Gateway Docs). It centralizes provider API keys behind one endpoint, supports traffic splitting and automatic fallback, and lets teams update routes without restarting the server. It supports fewer providers out of the box than the others here, but for teams already tracking experiments and models in MLflow, adding gateway routing to infrastructure you run anyway is the standout feature, not raw throughput.

Helicone: observability first, gateway second

Helicone is Apache 2.0 and TypeScript, and it functions as an AI gateway with unified access to 100+ models on top of its original job as an LLM observability platform, with request logging, prompt versioning, and cost tracking built in (Source: Helicone GitHub). The tradeoff is footprint: a full self-hosted Helicone stack needs Supabase (Postgres), ClickHouse, and object storage alongside the gateway worker, a heavier deployment than LiteLLM's own Postgres-plus-Redis baseline. Pick Helicone when observability depth matters more than a minimal footprint.

Comparison at a glance

GatewayOSS licenseSelf-host footprintStandout feature
LiteLLM Proxy (baseline)MIT core, enterprise/ separatePostgres + Redis, 2+ replicasBroadest provider coverage (100+)
BifrostApache 2.0Single Go binary, no DB requiredLowest claimed latency (vendor-reported)
Portkey AI GatewayMITnpx single command, no DB for base gatewaySmallest footprint, 250+ models
Kong Gateway (AI plugins)Apache 2.0Postgres or DB-less modeFolds into existing API infrastructure
MLflow AI GatewayApache 2.0Runs inside MLflow Tracking ServerNative fit for existing MLflow shops
HeliconeApache 2.0Supabase + ClickHouse + object storageDeepest built-in observability

The self-reported latency and throughput numbers for Bifrost and TrueFoundry's own gateway (not shown in this OSS-only table) come from each vendor's own README or blog, not a third-party benchmark (Sources: Bifrost GitHub, Portkey AI Gateway GitHub).

Operator note: what actually changes in your ops load

Operator note (first-hand): LiteLLM's own deployment guide recommends running 2+ replicas behind a load balancer, provisioning a managed Postgres instance, adding Redis once you scale past one instance, and pinning a specific image tag such as v1.90.2 rather than latest so rollbacks stay deterministic (Source: LiteLLM Docs). That is three pieces of infrastructure (app, database, cache) plus a version-pinning discipline to maintain. Compare that against npx @portkey-ai/gateway, which starts the base gateway with no database at all, or a single Bifrost Go binary with no external dependency for basic routing (Source: Portkey AI Gateway GitHub). If your team's actual complaint is "we maintain too much infrastructure for a proxy," the fix is not necessarily a faster gateway; it is a gateway with fewer moving parts.

That said, LiteLLM's Postgres and Redis are also what give it durable spend tracking and router state across restarts. A lighter gateway that skips a database also skips some of that persistence, so the tradeoff is real in both directions, not just an ops win for the lighter option.

How to choose

Match the gateway to infrastructure you already run rather than to a benchmark chart. If your team ships in Go and wants the smallest binary to operate, Bifrost is the natural fit. If you want the lightest possible footprint and are comfortable with a Node-based tool, Portkey AI Gateway's npx start and 122kb package size are hard to beat. Teams standardized on Kong for API management should look at Kong's AI plugins before adding a second gateway product. MLflow shops get gateway routing for close to free inside infrastructure they already run. And if observability, not routing speed, is the actual gap, Helicone's heavier stack buys real depth in logging and tracing.

None of these are a straight upgrade over LiteLLM in every dimension. LiteLLM still supports more providers than any single alternative here, and its Enterprise tier is a legitimate answer if SSO and audit logs are the actual requirement, not latency. The honest reason to switch is a specific operational limit you have already hit, not a chart on a competitor's landing page.

FAQ

Is LiteLLM open source?

Yes. LiteLLM Proxy's core, including the OpenAI-compatible gateway, virtual keys, spend tracking, and fallbacks, is MIT licensed. The separate enterprise/ directory in the same repository carries its own license and gates features like SSO past five users and audit logs (Source: LiteLLM Docs).

What is the difference between LiteLLM free and enterprise?

The free, MIT-licensed core covers the gateway, budgets, virtual keys, and logging. The paid enterprise tier adds single sign-on beyond five users, role-based access control, audit logs, and moderation integrations like Lakera prompt-injection detection, aimed at teams running 100+ users or 10+ production use cases.

What are the best LiteLLM alternatives for self-hosting?

Bifrost, Portkey AI Gateway, Kong Gateway's AI plugins, MLflow AI Gateway, and Helicone all support self-hosting under permissive licenses. The right pick depends on your existing stack: Go teams lean Bifrost, Node teams lean Portkey, Kong users add Kong's AI plugins, MLflow users use its built-in gateway, and observability-heavy teams pick Helicone.

Are there Python-first LiteLLM alternatives?

MLflow AI Gateway is the most Python-native option here, running inside the MLflow Tracking Server that most Python ML teams already operate. LiteLLM itself is also Python at its core, so teams staying in Python but wanting a lighter footprint are choosing between MLflow's gateway and accepting LiteLLM's existing Postgres-and-Redis requirement.

Are there Go-based LiteLLM alternatives?

Bifrost is the primary Go-based option, built by Maxim AI and released under Apache 2.0, with 23+ supported providers and no required database for basic routing. It is the closest thing on this list to a single-binary replacement for LiteLLM Proxy.

Related coverage

References