When to Use LiteLLM Self-Hosted vs a Managed AI Gateway

On September 3, 2026, ChatGPT, Claude and Grok all had outages inside the same few hours. Claude was down for three hours and six minutes. Everything was back by 12:38pm Pacific, and the Hacker News thread asking why three providers broke at once ran to 688 comments.

If your app called one provider directly, your app was down too. If it went through a gateway with a fallback configured, it wasn't.

So the gateway question isn't really "should I have one". It's who runs it. And the answer people reach for, self-host it because the software is free, is wrong more often than it's right.

What Does a Gateway Actually Do?

One API in front of many providers, plus the things you'd otherwise build twice. Automatic fallback when a provider errors or times out. Spend caps per key so one runaway loop can't empty your account. Request logging you can search when a customer says the output was wrong. Caching for repeated prompts.

LiteLLM's open source proxy gives you all of that for free. 100 plus providers behind one OpenAI-shaped API, virtual keys, users and teams, spend tracking, budgets, rate limits, fallbacks, request and response logging, Prometheus metrics. It's a good piece of software and the licence costs nothing.

Worth saying plainly: a fallback only saves you if the backup provider serves a model you'd accept. Falling back from Claude to a model that can't follow your prompt format turns an outage into a stream of bad output, which is sometimes worse than an error your app can retry. Pick the second choice deliberately and test it before you need it.

That's the part everyone gets right. The next part is where the money actually is.

What Does Self-Hosting Really Cost?

Not much, in hardware. A Hetzner CX22 is €3.79 a month before VAT for 2 vCPUs, 4 GB of RAM, 40 GB of disk and 20 TB of traffic. That runs a LiteLLM proxy for a solo SaaS without breathing hard. On Railway you'd be at $5 a month on Hobby including $5 of usage credit, more once Postgres and Redis are running.

You need both of those, by the way. Postgres holds keys and spend records. Redis handles rate limiting and caching. At small scale they sit on the same box happily enough.

Then comes the actual cost. A gateway sits in the request path, which means its uptime is your uptime. Every AI feature you ship goes through it. When it falls over at 3am, your product is down and you are the on-call rotation. You've taken a thing that was somebody else's problem and made it yours, to save roughly the price of a coffee.

If you already run infrastructure and enjoy it, that trade is fine. Our Vercel, Railway and Hetzner comparison covers the same instinct applied to hosting generally. But price it honestly, because the server was never the expensive part.

What Does Managed Really Cost?

Less than the self-hosting crowd assumes, which is what makes this decision lopsided.

Cloudflare AI Gateway's core features are free on every plan, including dashboard analytics, caching and rate limiting. Logpush costs $0.05 per million requests above the 10 million base, and unified billing adds a 5% fee on credit purchases. For a solo builder that's effectively zero.

Read that table again with the self-hosting argument in mind. The pitch for running your own is cost, and the cheapest managed option costs nothing while somebody else carries the pager.

So When Does Self-Hosting Actually Win?

Three situations, and they're all about constraints rather than price.

Data residency and air-gap. If prompts can't leave your infrastructure, no hosted gateway works, and this is the reason that survives every argument. LiteLLM Enterprise supports air-gapped deployment for exactly this.

Routing logic nobody sells. Model choice by customer tier, a bespoke cost ceiling per tenant, a fallback order that depends on your own data. Hosted products express the common cases well and the unusual ones not at all.

Log volume. Per-log pricing is fine until it isn't. Portkey Production includes 100,000 logs for $49 and charges $9 for each additional 100,000. Run 3 million logs a month and you're at $49 plus 29 blocks of overage, which is $310. Against a €4 server that finally reads like a real argument, and it's the point where most teams should switch.

Getting there takes a while, though. Three million logs a month is roughly 100,000 requests a day, every day. Most solo products never see that, and the ones that do have revenue to match.

Notice what's missing. "I want to save $49 a month" is not on the list, because at that scale you're spending far more on tokens than on the gateway, and the gateway is the cheapest line on the invoice.

What About Portkey's Acquisition?

Palo Alto Networks completed its purchase of Portkey on May 29, 2026, and Portkey is now the AI Gateway inside Prisma AIRS, aimed at securing AI agents. The press release says nothing about the standalone product, the open source gateway, or what happens to existing pricing.

Silence isn't a reason to migrate this week. It is a reason to keep your setup portable. Any of these gateways speaks the OpenAI API shape, so switching should be a config change rather than a rewrite, and it stays that way only if you don't build against one vendor's proprietary features. The LiteLLM, Portkey and Cloudflare comparison goes deeper on the feature differences.

The Pick

Start managed. Start with Cloudflare AI Gateway, because free and operated by someone else beats €4 and operated by you, and it takes one line of code to put in front of your existing calls.

Move to Portkey or Helicone when you want better logging and search than the free tier gives you, somewhere around the point where you're debugging customer complaints weekly rather than monthly.

Self-host LiteLLM when a constraint forces it. Compliance, air-gap, or routing you can't buy. Not to save money, because you won't.

And whichever you run, set the spend caps on day one. A gateway without budgets is just a faster way to discover what a runaway agent loop costs overnight, and configure the fallbacks too, because September 3 will happen again. The OpenRouter alternatives roundup covers who to fall back to.