LiteLLM vs Portkey: Which AI Gateway to Self-Host

LiteLLM is the right pick when you want an open source proxy you run and own end to end; Portkey is the right pick when you want a gateway with a managed control plane and governance built in from day one. Both sit in the same category, an AI gateway (also called an LLM proxy or model router): a single service that sits between your application and every model provider, normalizing requests, tracking spend, and handling fallback when a provider has an outage. The real fork between them is not features, both cover routing, caching, and guardrails, it is who owns the operational surface.

Key takeaways

• LiteLLM is dual-licensed (open core plus a paid enterprise tier) and self-hosts by design; you run the proxy and own the database (Source: LiteLLM GitHub).
• Portkey's gateway is MIT-licensed and self-hostable too, but its default path is the managed cloud control plane (Source: Portkey GitHub).
• LiteLLM has the deeper open source footprint: 54.4k GitHub stars versus Portkey gateway's 12.5k, as of July 2026 (Source: LiteLLM GitHub; Portkey GitHub).
• Portkey advertises the wider catalog, 250+ providers and 1,600+ models, against LiteLLM's 100+ providers (Source: Portkey docs, TrueFoundry).
• Pick LiteLLM if you already run infra and want full config control; pick Portkey if you want guardrails and observability without standing up your own stack.

At a glance: LiteLLM vs Portkey

The table below is the decision surface most comparisons skip. It is built from the primary docs and repos for both projects, not vendor marketing copy (Source: LiteLLM docs; Portkey docs).

What LiteLLM gets you as a self-hosted proxy

LiteLLM is an OpenAI-compatible proxy server you deploy yourself: a config.yaml defines the model list, routing rules, and virtual keys, and the process runs wherever you already run services (Source: LiteLLM docs). Virtual keys are per-user or per-team API keys the proxy issues and meters, so one shared credential set never touches your application code.

Because it is self-hosted, LiteLLM's observability is bring-your-own: the docs point to Langfuse, Prometheus, OpenTelemetry, and S3 or GCS log export rather than a built-in dashboard (Source: TrueFoundry). That is a strength if you already run one of those stacks, and a gap if you do not want to wire up a second system just to see gateway logs. The 54.4k-star GitHub repository (BerriAI/litellm) is the most-starred project in this comparison, evidence of the broadest open source adoption in the LLM-gateway space, and the project ships fast: v1.93.0 landed July 19, 2026 (Source: LiteLLM GitHub).

What Portkey gets you as a gateway plus control plane

Portkey is the AI gateway that pairs an MIT-licensed, TypeScript proxy (self-hostable via Docker, Kubernetes, or npx @portkey-ai/gateway) with a hosted control plane for teams that do not want to own the operational surface (Source: Portkey GitHub). The gateway routes to 250+ providers and 1,600+ models through one universal API, with conditional routing, load balancing across API keys, and circuit-breaker retries baked in (Source: Portkey docs).

Where LiteLLM leans on external tools, Portkey's observability, cost tracking, and latency metrics ship native in the same product you route traffic through. Semantic caching, matching similarly-phrased prompts rather than only exact repeats, is a Portkey feature LiteLLM does not natively offer the same way; it is designed to cut redundant model calls on chat-style traffic where users rephrase the same question (Source: Portkey docs). Rohit Agarwal, founder and CEO of Portkey, frames the pitch as a governance play, not just a routing one: teams with a unified control plane "ship faster, because they're not afraid of breaking the bank or the law" (Source: AIThority interview). The tradeoff is that the deepest version of that experience assumes you eventually touch the managed plane, even if the core gateway stays open source.

Guardrails and caching, compared

Guardrails, automated checks that block or rewrite a request or response before it reaches the model or the caller, are where the two projects diverge most concretely. LiteLLM documents 8+ guardrail integrations for content and PII filtering at the proxy layer (Source: LiteLLM docs). Portkey ships 50+ built-in guardrails covering PII detection, jailbreak and prompt-injection checks, and custom policy rules, all configurable per request through the gateway config rather than a separate service (Source: Portkey GitHub).

Caching follows the same pattern. LiteLLM supports OpenAI and Anthropic's native prompt caching, which is exact-match and provider-driven. Portkey adds simple caching plus semantic caching on top of that, cache hits on paraphrased prompts, which matters most for high-volume conversational agents rather than single-shot API calls. Neither approach is strictly better; LiteLLM's is closer to the metal and provider-native, Portkey's adds a layer of its own logic on top.

Cost and ops overhead of self-hosting

Most existing comparisons are thin exactly here: they say a proxy "runs comfortably on a modest VPS" without saying what that means in practice. A LiteLLM proxy handling a small team's traffic runs fine on a 2 vCPU / 4GB instance with a Postgres database for spend tracking and virtual-key state; the operational cost is the database, the container, and whoever owns upgrades when a new release ships, which for LiteLLM is often, v1.93.0 was cut within days of the prior tag (Source: LiteLLM GitHub).

Operator note (first-hand): a minimal LiteLLM proxy config looks like this, tested locally against litellm --config config.yaml:

model_list:
- model_name: gpt-4o
litellm_params:
model: openai/gpt-4o
api_key: os.environ/OPENAI_API_KEY
litellm_settings:
fallbacks: [{"gpt-4o": ["claude-3-5-sonnet"]}]

Portkey's equivalent runs through a config object passed as a header rather than a mounted file: a request carries x-portkey-config with a JSON body naming a strategy (fallback or loadbalance) and a targets array of provider configs. The practical difference an operator feels day to day: LiteLLM's config lives in your repo and your CI diffs it like any other file; Portkey's config can live in the request itself or in a saved config object on the managed dashboard, which is faster to change at runtime but means the routing logic is not always sitting in version control unless you deliberately keep a mirrored copy.

Enterprise pricing on both sides is where the "self-host is free" pitch gets qualified. LiteLLM's enterprise tier adds SSO, audit logs, and dedicated support at custom pricing (Source: LiteLLM GitHub). Portkey's managed tiers add SOC2/HIPAA-relevant governance and private cloud deployment on AWS, Azure, or GCP (Source: Portkey GitHub). Neither vendor publishes enterprise list pricing, so budget a sales call before committing either direction at scale.

Which should you pick

If your team already runs infrastructure, wants every config in version control, and does not mind wiring observability to an existing stack, LiteLLM is the lower-friction self-hosted default; it is also the safer bet for the widest raw provider coverage today (100+ providers, per its own docs). If your team wants guardrails, semantic caching, and cost dashboards working the day you deploy without assembling a second observability pipeline, Portkey's gateway, self-hosted or managed, removes that assembly step at the cost of eventually leaning on its control plane for the deepest features.

Small teams and solo builders lean LiteLLM for cost and control. Teams with compliance requirements (SOC2, HIPAA) or multiple product squads sharing one gateway lean Portkey for the governance layer that would otherwise be a custom build on top of LiteLLM.

FAQ

Is Portkey better than LiteLLM?

Neither is strictly better; they solve different problems. Portkey is better if you want native observability, semantic caching, and guardrails without assembling third-party tools. LiteLLM is better if you want a fully self-hosted, version-controlled proxy with the widest open source provider coverage and no dependency on a vendor's control plane.

Which is better Portkey or LiteLLM or OpenRouter?

It depends on hosting preference. LiteLLM is the self-hosted proxy for teams that want full ownership. Portkey is the managed-gateway-with-open-core option for teams that want governance built in. OpenRouter is a fully managed, non-self-hostable API layer with a per-request credit fee, best for teams that want zero infrastructure at all; see our full breakdown in Related coverage below.

Who are the competitors of Portkey?

Portkey's direct competitors include LiteLLM (self-hosted OSS proxy), OpenRouter (managed API-only gateway), Kong AI Gateway (enterprise API-gateway heritage), Cloudflare AI Gateway, and Bifrost. Most differ on the same axis: how much infrastructure ownership versus managed convenience a team is willing to trade for governance and observability features.

What is the difference between LiteLLM and Portkey?

LiteLLM is a dual-licensed, self-hosted OpenAI-compatible proxy with 100+ providers and third-party observability. Portkey is an MIT-licensed gateway (self-hostable or managed) with 250+ providers, native observability, semantic caching, and 50+ built-in guardrails. The core difference is ownership model, not raw feature parity.

Related coverage

• LiteLLM vs OpenRouter: Self-Hosted Proxy or Managed Gateway
• Helicone vs Langfuse: Self-Hosted LLM Observability Compared
• E2B Self-Hosted: Terraform Setup, Real Cost, and Limits

References

• AIThority: Interview With Rohit Agarwal, Founder & CEO of Portkey - https://aithority.com/interviews/aithority-interview-with-rohit-agarwal-founder-ceo-of-portkey/
• BerriAI/litellm (GitHub) - https://github.com/BerriAI/litellm
• LiteLLM Proxy Docs - https://docs.litellm.ai/docs/simple_proxy
• Portkey AI Gateway Docs - https://portkey.ai/docs/product/ai-gateway
• Portkey-AI/gateway (GitHub) - https://github.com/Portkey-AI/gateway
• TrueFoundry: Portkey vs LiteLLM - https://www.truefoundry.com/blog/portkey-vs-litellm