Whoever has your API keys has your AI

In an attack on LiteLLM, a tool that bundles requests to different AI models, more than 2500 companies were affected. The interesting part is not the tool. The interesting part is the position. A broker like that sits exactly where the credentials have to sit for it to do its job. Whoever holds that position is you, as far as the model provider can tell.

An API key is not a password with a second factor and a login log. It is a string, and the string is enough. There is no device that confirms on top, and no follow-up question. That is why the question of where this key sits matters more, when you use AI, than the question of which model is currently the best one.

What actually leaks in a case like this

Not the model. That belongs to the provider anyway. Three other things can leak.

First, money. Somebody else's key gets resold and metered, and the invoice lands with you. Second, the content. The requests to a model carry your operating material, so customer enquiries, text blocks from quotes, prices, internal notes. Third, the way you work. Simon Willison has written down why even a model's reasoning traces are a target in their own right. Anyone reading along sees not just results, but also the instructions you use to produce them.

That is no reason to avoid cloud AI. It is a reason to treat the credentials like cash and not like a line of configuration.

How we handle it here

We run our own infrastructure ourselves, so cloud, mail server, password manager, Git server and this website. For Kunst gegen Bares, the platform for the live art stage in Düsseldorf that we built and operate, two external connections come on top. One for the AI texts, one for publishing to the channels. Both leave the house, and we say so openly, because the last mile cannot work any other way. If you want to publish on Instagram, you send data to Instagram.

What we can control about it are the keys. Four building blocks.

• No key in the source code. Credentials are not in the Git server and not in an application directory that gets copied along. They sit in a place the service reads them from at runtime, and that place is handled differently in the backup design than everything else.
• One key per purpose. The connection that produces text suggestions is not the same one that is allowed to publish. If one of them looks wrong, exactly one part goes down and not the whole chain.
• A log. It is traceable which service used which connection and when. Without that, after an incident you can only guess, and guessing helps nobody.
• Rotation as a rehearsed procedure. Create a new key, store it, reload the service, revoke the old one. Written down, tried out at least once. A key you could swap in theory does not get swapped when it matters, because at that moment nobody knows what will stop working afterwards.

Unspectacular, and dependable for exactly that reason.

Interchangeable is a security feature too

Freedom of choice on the model is usually discussed under the heading of independence. But first of all it is an operational question. If a connection is compromised, you want to be able to switch it off without the application coming to a standstill. That only works if the model connection is an interchangeable building block and not baked into fifty places in the code.

How deep you go is your decision. There are levels.

A large cloud provider is a legitimate choice if keys and data flow are kept clean. A provider in Europe is the next level and usually costs a bit more per request. A model on your own hardware is the third, and honestly that one costs the most, namely purchase, electricity, updates and a model that is weaker at some tasks than the best one from the cloud. In return, nothing leaves the house for that step, and there is no key sitting with a third party.

What matters is that the level stays a decision and does not become a dead end. As long as the data and the system belong to you and the connection to the model is a piece of configuration, switching is a matter of hours.

Where a person decides

Revoking a key means halting part of the operation. We leave that decision to no automation. Monitoring shows what looks unusual, for instance requests at times when nobody is working, or consumption that does not fit the campaign plan. Whether something gets revoked, rolled back or just watched for now is decided by a person.

The same goes for the payload. At Kunst gegen Bares the AI produces text suggestions per channel, and publishing happens after a person approves them. Anyone abusing the text connection gets drafts at worst and no posts, because publishing is a separate connection with a human approval in front of it. The machine suggests, a person approves. For us that is not a slogan about attitude but a build instruction, and in a security incident it pays off.

Five questions for your own house

You do not need a security project to make progress here. Five questions are enough to start with.

1. Who can read the key for the AI connection today, and which backups is it sitting in?
2. How long does it take to swap it, and has anybody ever done that?
3. Is there a spending limit at the provider, or is the invoice open ended?
4. Can you see after the fact which service sent requests and when?
5. What breaks if you have to change model providers tomorrow?

If you have no answer to the second and the fifth question, there is more risk sitting there than in the choice of model. And that is the good news, because this part is craft and not a matter of belief.