Millions of Vulnerable PHP Services in Germany, and Who Patches at Your Place on Sunday Evening

The order of events was backwards. Details on the php2shell flaws were making the rounds before the updates were ready, and attacks were already running when many administrators read about the problem for the first time. According to an analysis by Censys, millions of PHP services in Germany are vulnerable. And for the third flaw there is no emergency patch at all for now.

PHP runs under a great many company websites, shops and customer portals. So the interesting question is not whether this is bad. The interesting question is who at your company installs the update on Sunday evening, within what deadline, and what you do when there is nothing to install.

The list first, the excitement later

Most companies do not fail at the update itself. They fail because nobody knows where PHP is actually running. Take twenty minutes and write down what you really have out there on the network. At a machine builder with 60 people, that list often looks like this:

• the website with its content management system, looked after by an agency
• the spare parts shop, built four years ago, touched twice since
• the customer portal with the maintenance records
• a holiday and time tracking tool left over from the days before payroll was outsourced
• a test system from 2021 that was never switched off and is still reachable from the internet

For every line you need three pieces of information. Who maintains it, how do you reach that person outside office hours, and is the service publicly reachable or only on the internal network. In practice the test system from 2021 is the biggest risk, because it is in no contract and in nobody's head.

Two numbers you have to set

A deadline that is written down nowhere gets debated instead of met when things get serious. Set two numbers and put them into the contract with your service provider, or into the job description of whoever handles it internally.

The first number applies to critical flaws that are already being attacked. Realistically that is 24 hours from the moment the update is available, and rather less for a shop. The second number applies to everything else, where 30 days is a common value. What matters is not whether you pick 24 or 48 hours. What matters is that somebody knows the clock is running, and that this person carries a phone.

That this window does not close on its own is by now well documented. In a study, the Cyberagentur concludes that software vulnerabilities remain a problem for longer, meaning vulnerable systems are still on the network months after the patch. That matches what administrators report in weeks like these. The update itself takes minutes. What takes time is the sign-off, getting into the old server nobody has the key for any more, and the fear that the shop will not start up afterwards.

When there is no patch

Which is exactly the case with the third flaw. Then the work shifts from "install it" to "shrink the attack surface", and those are concrete interventions:

• switch off the affected function if you do not need it
• take the service off the internet and make it reachable over VPN only
• restrict access to known address ranges, for instance on the customer portal with twelve customers
• configure a filter in front of it that blocks the known attack patterns
• read the logs daily instead of weekly

The test system from 2021 you switch off. The shop two weeks before Christmas you do not. No software can make that call, because it depends on the business and not on the severity of the flaw. A person decides, and that person should know in advance that it is them.

Three ways, and the middle one usually gets overlooked

For day to day operation there are not two options, there are three.

Run it yourself. Your own machine, your own maintenance. The machine at a provider in Germany costs 40 to 80 euros a month, and that is never the problem. The problem is the person who is reachable on Sunday evening, and the stand-in for when that person is on holiday.

Have it run for you. The software runs on your server and under your contract, and somebody else maintains it. The data and the system belong to you, the service provider can be replaced, the on-call duty is bought in. This is the category we work in, and it is exactly how we run our own cloud, the mail server, the password manager and the Git server.

Rent it. You pay per head, the provider patches, you find out via a status page. That is a legitimate choice, especially for standard things. What you give up is control over the timing, and over what happens if the provider rates the flaw differently than you do.

Sovereignty here is a question of depth, not an either-or. A mixed setup is the normal case, and it is fine as long as it is clear for every line of your list which of the three categories it falls into. It only gets awkward with the services where all three answers are half true at the same time.

What may run automatically and what may not

Small security updates to the operating system can be installed automatically at night, with a backup beforehand and a notification afterwards. That takes the dull work away and in weeks like these it is the difference between hours and weeks.

In two places that stops. A major version jump, where data formats or configurations change, gets read, tried out on a copy and then approved by a person. And when monitoring reports that something is wrong, it shows what looks unusual. Whether to restart, roll back or just watch for now is decided by a person. That is how we build it, and that is how we handle it for the platform for Kunst gegen Bares, which we run ourselves.

Your Monday morning

Write the list. Put a name and a phone number behind every line. Set the two deadlines. Call the provider who looks after your website and ask whether php2shell is dealt with at your end and when. Switch off whatever nobody needs any more.

That takes a morning, and it holds again every time the next flaw leaks ahead of the fix. And it will.