When Nobody Reads What the Machine Wrote

A developer describes his working day at a large corporation like this. Spec, tickets and code come entirely from an AI agent, and none of it gets read anymore. Simon Willison quotes this report on his blog. It is not a doomsday scenario, it is a description of daily operations, and it is uncomfortable precisely because it sounds plausible.

Two other pieces of news land in the same period. Microsoft had agents port a runtime to Rust, and the bill came to around 120,000 dollars. And Anthropic is turning its Projects into a task dispatcher that sets several agents on tasks in parallel. Both increase the number of results per day. Neither increases the number of people who look at them.

The port shows what actually matters

The interesting part of the Rust port is not the price, it is the type of task. With a port, the old system still exists. There is a behaviour you can hold the new result against, and if tests exist, they run against both sides. A machine working here works against a yardstick it did not invent itself.

That is the difference to an agent that writes a spec, derives tickets from it, builds the code to match and finally the tests that confirm the code. Then the yardstick comes from the same source as the result. If a system outputs a number and nobody can trace where it came from, the number is worthless. The same goes for code, it just takes longer to show.

For you as the client that means one thing. Don't ask whether AI was involved. Ask what the result was measured against and who set that yardstick.

How you can tell in an acceptance meeting whether anyone looked

A person who has genuinely reviewed a result has found something in the process. Always. That is why the most revealing question in an acceptance meeting is not "does this work?" but "what doesn't work yet?". Someone who names three known gaps and tells you which of them will hit you in daily operation has read what was built. Someone who says it all runs fine has had it demonstrated to them.

The second test is a decision. Pick a spot where the solution looks different from what you expected and have it explained to you. A considered answer sounds like a trade-off, something like "we tried X first, that failed because of Y". An unchecked answer sounds like a description. It tells you what the code does, not why it is built that way.

Four questions for your service provider

1. Who read this, and how do I see that? A name, a date, a review entry. Not "our team".
2. What did generating it cost, and what did checking it cost? Listed separately. If the review does not appear as a line item, it never appeared in anyone's calendar either.
3. What happens if I deliberately break a test? In other words, is there an automated check that actually trips, or one that only ever glows green? That can be demonstrated in ten minutes.
4. Who explains this part to me in eighteen months? An agent run is rarely reproducible. What remains is code, tests and documentation. If those three don't line up, the solution no longer belongs to anyone.

The fourth question is the one that costs the most and gets asked the least. Software does not get expensive while it is being built. It gets expensive when someone has to change it who wasn't there.

Write the sign-off point into the workflow, not into the presentation

Agents working in parallel move the bottleneck. When five tasks finish at the same time, the scarce resource is no longer the output, it is attention. That is exactly where the habit of merely skimming a result takes hold. No amount of good intentions helps against that, only a point in the workflow where nothing moves on without a click.

Make that concrete for one of your own processes. If a customer enquiry by email automatically turns into a draft quote in your ERP, then the sign-off point is the moment before the quote leaves the building. Define who clicks, what they look at as a minimum, say quantity, price and delivery date, and that their name and a timestamp stay in the record. Then check the rest of the workflow with a single question. Is there a second step where a mistake goes out the door or money moves? Then that one needs a click too.

And if nobody can be found to do a sign-off because everyone has too much on, that is your answer. Then this step does not belong in the automation yet, it stays with a person until someone has the time for it in their calendar.

In the social automation for Kunst gegen Bares, which we built and run ourselves, that point sits in the calendar. The AI writes the text variants for each channel, and nothing goes out until a person has approved the post. It costs a minute per post, and it is the reason we can hand the drafting over to the model that freely in the first place.

What to take away

The news of these weeks does not show that agents can do too much. It shows that the bottleneck has moved, and that organisations tend to walk past a bottleneck rather than staff it. Your job as a client is small and unspectacular. Demand a name on every result, a separate line for review in the quote, and a named point where someone from your own company signs off. That can be settled in one conversation, and it lasts longer than any assurance about which tools were used.