What I found by asking teams how they handle runaway AI spend and the one question almost none of them could answer.
Over the past few months I've been asking a very specific question: how do teams running LLMs in production actually handle runaway spend, and what happens when the prompt data can't leave their environment?
I promised everyone who took part that I'd share what I found. So here it is. Be clear about the sample before you weight any of it: three long conversations with founders and CTOs, a public thread with a dozen or so responses and desk research across 24 suppliers on the NHS ambient-voice registry. And to be clear - where I'm hypothesising, I say so.
This was the most consistent finding and the one I didn't expect to be so lopsided. Every sophisticated team I spoke to had built cost visibility i.e. token-to-cost mapping, per-agent daily spend, internal dashboards, custom alerting etc... Several had built it twice. One described rolling their own as straightforward work that any competent team could do, which in this day-and-age they're right.
Almost nobody had built enforcement - something that refuses the request before the money leaves the business account. Runaway spend lives in the gap between noticing and stopping, and that gap is usually measured in hours of human attention. One founder described two internal bots that ended up pointed at each other:
"I suddenly saw a spike. I went and investigated and then I shut it down."
Founder, logistics AI
That worked because he happened to look. The same story at 2am, or over a bank holiday weekend, ends differently. When I asked what would stop an agent loop overnight, the honest answer was almost always a person noticing in the morning.
This one is underrated, I think, and it was raised independently by several people. Microsoft, Google and AWS are handing out substantial credits. Teams burning real money on inference aren't feeling it yet, because they aren't paying for it yet.
"We just got a new tranche, so it's sometime in 2027, but then it would just hit our books at that point."
Co-founder, insurance AI (spending $15-20k/month)
The pain isn't absent. It's scheduled. That has a strange implication: a lot of teams are going to discover cost governance on roughly the same date, when their tranches run out and a line on the accounting books that was free becomes a line the CFO reads. If you're on credits, the useful exercise is to find out when they expire and what your run rate looks like the day after.
That's a direct quote about data sovereignty from a founder whose product sends customer data through third-party models. It stuck with me because I heard versions of it repeatedly and because it goes against the prevailing narrative that data residency is a blocking concern for everyone.
Digging into why, the pattern seems clear. Most teams have an escape route from the constraint where they either:
Any one of those removes the problem. The constraint is only an issue for teams where none of them are available i.e. where the sensitive content is the payload, it can't be anonymised and still be useful, and the task needs frontier-model quality. A clinical consultation is the clearest example I found: you cannot redact a patient conversation into something a scribe can still summarise before sending it to a frontier provider.
If you're trying to work out whether residency is a real constraint for you or a background worry, that's the test. Which of the three exits can you take?
I went in assuming hard spend caps were obviously good. The sharpest pushback I got was from a founder running customer-facing workloads:
"We still need it to go through. Blocking it would make our customers upset."
Co-founder, insurance AI
He's right and it's the thing I'd most want anyone building in this space to know. For mission-critical serving traffic, refusing a request can be worse than the overspend it prevents. Any usage cap that could possibly fire during normal operation is a liability, so keep it in mind when architecting the systems.
Where I landed, after sitting with it for a bit: the useful mental model isn't a budget, it's a circuit breaker. A breaker doesn't trip when you run the kettle and the toaster at the same time - it trips when something is genuinely wrong with the system. A ceiling set near expected usage is a sledgehammer. A ceiling set at 5x or 10x expected usage never touches legitimate traffic and only exists for the agent loop nobody intended to let runaway. Also, it matters massively where you put it: the customer-facing path and the experimental batch job should not share the same limit. Most of the risk lives in the places you'd happily stop e.g. experimental workloads, dev environments etc...
Several people assumed their provider's dashboard had this covered. It's worth checking rather than assuming, because the controls vary a lot. I did, and here's what I found:
None of that makes provider controls useless. It just means "we've set a cap in the console" and "our spend is bounded, so we're good" aren't the same thing.
If you work in health tech: I've found that the ambient-scribe market splits into engines and vendors. Several suppliers on the NHS registry are reselling another supplier's engine under their own brand and security review. Only the engines carry the model bill, so the unit economics of AI scribing are concentrated in far fewer companies than the supplier list would suggest.
A pattern across every conversation: the more sophisticated the team, the more likely they'd already built their own. Less of a product decision, more of a reflex. One CTO summed up the economics of buying anything too specific:
"I'd rather have a tool that did more things and solved the full problem for me than just one thing."
CTO, edtech
The same person, unprompted, drew the line I'd expect: "For larger organisations this is going to be a concern. For startups, maybe not that much." Which matches everything else I'd found - this is a problem that becomes real when someone other than the person who built the system is accountable for what it spends.
The question nobody I spoke to could answer cleanly: what does one customer cost you? Everyone can see the total bill. Almost nobody can decompose it per customer, per team, or per feature and for anyone whose AI spend is the cost of goods sold, that's the number that determines whether the business works. One team described their per-customer cost metric as an ongoing experiment. Several others hadn't started.
If I were picking a problem in this space today, it would be that one.
Four things, if you're running LLMs in production:
Full disclosure, as I gave everyone I spoke to: I built a thing in this space - a self-hosted proxy that enforces spend limits before requests leave your network, without holding your API keys or touching the payload. The research was meant to tell me whether there was a business in it.
The honest answer is that I never found out. I couldn't reach enough of the ideal consumer profile to know and for now I've parked it rather than keep guessing. But the findings above cost other people their time, so they belong in public rather than in my notes. If any of this is useful, or you think I've got something wrong, I'd like to hear it. Or if you'd like to continue discussions about my solution, I'd like to hear that too.