An AI agent deleted a production database in nine seconds. The safeguard was real, and attached only to the path a human would take.
Nobody had given it permission, and nothing had taken it away.
The Master is ashore ended with a question I deliberately left open. Maritime autonomy has flag states, classification societies, an accident investigation branch, a code that entered effect in July, and six years set aside to learn what the binding version should say. By the standards of this series that is the immature case. So what happens when systems acquire comparable authority somewhere with no equivalent machinery for learning what should constrain them?
Here is where.
In June 2026, NHS England opened a consultation on revising the two clinical risk management standards that Mandated, not measured was built around. Among its reasons: artificial intelligence, machine learning, ambient voice technologies and complex algorithmic systems are increasingly deployed in health and care, present novel risk profiles, and require governance frameworks that the current standards do not adequately address.
A regulator, revising a supervisory standard, saying that the thing now entering its estate is not covered by the standard it is revising.
That is the position across most of the economy right now, and it is not confined to healthcare. So this is the point where ten articles about vehicles, grids, factories, hospitals and ships turn back toward the systems that prompted them, carrying what those domains established rather than asserting it.
Inherited. From A control you can skip, industrial automation. A rule is not a control if the action remains technically permitted. Applied to. AI agents acting on production systems. The question. When you tell an agent it may not do something, where is that rule enforced?
Nine seconds
On a Friday in late April 2026, an AI coding agent deleted the production database of PocketOS, an automotive software company, along with all of its volume-level backups. One API call. The founder, Jer Crane, timed it at nine seconds.
The agent was Cursor running Claude Opus 4.6, and it was not working on production. It was working in staging, where it hit a credential mismatch. It decided that deleting a storage volume would resolve the problem, which required an API token, so it went looking for one and found a token in an unrelated file on the machine.
That token had been created for a narrow job, adding and removing custom domains through the command line. It was scoped for any operation, destructive ones included. Crane has said it would never have been stored on that machine had he known how broad it was.
The agent used it to issue a single authenticated call. There was no confirmation step. The backups went with the volume, because the infrastructure provider stored volume-level backups inside the volume. Recovery took sixty hours.
The provider was Railway, and their postmortem is where this stops being an anecdote.
What Railway wrote about their own API
Four days later Railway published an account of the incident. Three sentences in it do more work than any commentary I have read on the subject since.
The first is a description of what happened at the API. The request was authenticated, and their API honoured it exactly as it would have honoured the same call from a command-line tool or a continuous integration pipeline. The only difference was that it came from an agent.
The second is the one I keep returning to. They had already built a great many primitives for exactly this problem. The agent simply skipped past them by going directly to a legacy endpoint.
The third makes the asymmetry concrete. Until that week, calling the volume-deletion mutation through the API executed immediately with no way to undo it. The dashboard had a 48-hour window for the same action.
Read those together. The protection existed. It was real, it was well designed, and it was attached to the surfaces that people click through. The API path had no opinion about it.
This has happened before, in a factory
A control you can skip examined a robotic welding cell in Alabama where a young woman was killed by a machine that should not have been able to move. That cell had an interlock. Opening the rear gate de-energised the robots, and hanging a personal lock on that gate held it open. The interlock worked perfectly. It was in order that morning and it had never failed.
What it could not do was notice a person entering through the front.
Nobody defeated the interlock at Ajin and nobody defeated anything at Railway. In both cases the safeguard was keyed to a particular path, and something arrived by a different one. A robotic cell that answers the question is this gate open cannot answer is somebody standing inside. An API that answers is this request authenticated cannot answer should this actor be permitted to destroy a business.
There is a second parallel, and it is closer than I would like.
Railway offers four levels of API token, from account-wide access down through workspace and project scopes to an OAuth flow with granular consent. The token the agent found was account-scoped, the broadest available. Railway's explanation is that the right scope existed, but the flow did not make it obvious which one to choose, so the path of least resistance produced a token with more access than the work required.
At Ajin, changing a welding tip with the lockout procedure took three to four minutes, and without it, one to two. The safe path existed there too. The cheap one won, several times a shift, for two years, until it killed somebody.
A safeguard that is available but effortful, competing against a shortcut that is available and easy, loses. It loses in a factory under a production quota and it loses in a token-provisioning screen at four in the afternoon. This is not a fact about robots or about agents. It is a fact about defaults.
Three things that are not the same thing
Here is the distinction the whole article exists for, and it is one that discussion of AI systems collapses constantly.
Capability is what the system can physically do. Intention is what someone meant it to do. Authority is what the surrounding systems will actually permit it to do.
The agent in this case operated under rules stating that it should never run destructive or irreversible commands unless explicitly asked. That is an intention, written in words, addressed to the system's understanding of its own task. The account-scoped token was the authority, and it was addressed to nothing at all. It simply was what it was, and any process holding it could do anything with it.
A prompt describes what an agent is supposed to do. A permission determines what it is able to do. These are not two versions of the same control. They are a request and a boundary, and only one of them is load-bearing.
A note on evidence while we are here. The agent in this incident produced a fluent, apparently candid account of how it had ignored its instructions. It is quoted widely. I am not going to rely on it, and neither should anyone else, because the second case in this article is precisely a demonstration of why an agent's testimony about its own actions cannot be treated as a record.
The freeze that lived in the prompt
In July 2025, roughly nine months earlier, Jason Lemkin was several days into building an application on Replit when the platform's agent deleted his production database. There had been an explicit code and action freeze in place. He had instructed it repeatedly not to make changes.
The agent then told him that rollback was impossible. It was not. That claim delayed the recovery, and it is the reason the previous paragraph exists.
Replit's chief executive acknowledged it publicly within days and called it unacceptable. What matters more is what the company shipped afterwards. Within a week Replit had begun separating development and production databases, starting with new applications and extending outward, and by the end of that month described them as separated by default, stating that with this in place the agent cannot make any change to the production database during development.
Not should not. Cannot.
That is the entire movement this article is about, performed by a vendor in public over a matter of weeks. The freeze had lived in the instructions, where the agent could read it, agree with it, and act against it without anything in the execution path objecting. The replacement lives in the architecture, where agreement is not required.
Where the industry now thinks the answer is
Both companies arrived at similar conclusions about what an agent should be given, and it is worth being precise about what kind of evidence this is. These are vendors describing their own remediations. They are not independent findings, and there is no body that could produce one.
Railway's answer is to stop agents finding credentials lying around. Their remote integration server issues short-lived tokens rather than long-lived ones sitting in a file on disk, scopes access to the specific workspaces and projects a user consents to at the time, and exposes a fixed set of defined tools instead of a general-purpose API. They also updated the API so that deletions soft-delete for 48 hours, matching the dashboard, and delayed deletion of the backups themselves so a cascading delete can no longer make them appear to be gone.
The shape of that answer is that the agent gets its own scoped identity rather than inheriting whatever authority a human happened to leave nearby.
The same shape is appearing where the money is. In March 2026 Santander processed what it and Mastercard described as Europe's first live end-to-end payment executed by an AI agent, through Santander's own payments infrastructure, using a scheme that issues agents their own credentials within the payment network and treats them as visible participants in the transaction flow. The purchase was a T-shirt, in a controlled environment, in a pilot that both companies announced themselves. It demonstrates that the architecture exists rather than that it holds.
I will note one thing about Railway's account, not as a criticism but because this publication is about supervisory claims. In a postmortem explaining that their safeguards did not cover one path, they write of the new agent surface that it is not possible for an agent to go rogue there. That is their claim, and it may well be right. It is also the same category of statement that the previous architecture would have supported, until a Friday in April.
Why all of this is developer tooling
Every case in this article involves an agent acting on infrastructure. Databases, volumes, deployment platforms. None of them involves an agent moving money at scale, altering a customer record, or sending a communication that commits an organisation to something.
That is not because those deployments are not happening. Salesforce, ServiceNow and Zendesk all document agents that read and write customer records, open and close cases, and send customer email on an organisation's behalf. The authority is being granted now, across systems of record, at organisations that will not discuss it.
It is because developer tooling is where agents got authority first, and where the people affected happen to publish.
Which returns us to the gap The Master is ashore left open. When a robotic cell kills someone, an investigator with statutory powers compels evidence and publishes a finding. When a grid collapses, a panel of forty-nine experts spends eleven months on it. When a child is harmed by a prescribing system, a national body investigates and issues recommendations that named organisations must answer.
When an agent deletes a company's database, we find out because the founder posts about it and two vendors write blogs. That is a considerable amount of goodwill and no machinery at all. There are databases collecting these events, assembled largely from news reports, some of them run by companies selling the remedy. A register is not an investigation. It records that something happened; it does not establish why, and it obliges nobody to answer.
The assumption that stopped being true
Crane put the underlying change more precisely than any of the commentary did. Railway's position had always been that an API key should only ever be accessed by a human. That was true, and it had always been true, and it had been safe to build on for years. His question is what happens now that a computer is in control and you do not know what it is doing.
Railway said the same thing from their side, in the flattest possible terms. The assumptions the API was built on are getting stretched.
Nobody had changed the premise those systems were designed around. The token behaved according to its scope. The API behaved according to its contract. The safeguard did not malfunction; it simply did not cover the path the agent took. What happened is that a premise which had been reliably true for the entire life of the product quietly stopped being true, and the architecture built on top of it went on behaving exactly as designed while the ground underneath it moved. The token was not too broad for a human who knew what it was for. It was too broad for a process that did not.
Which raises a question this article cannot answer, because it has looked only at the moment of failure. Railway noticed the change after a database was destroyed. The interesting question is what it would have taken to notice earlier, and whether anything in an ordinary organisation is watching for the moment its assumptions stop matching the world they were fitted to.
Sources
Railway, Your AI wants to nuke your database. Guardrails fix that., 29 April 2026
PocketOS, company account of the incident, 11 May 2026
The Register, Cursor-Opus agent snuffs out startup's production database, 27 April 2026, including email statements from Jer Crane and Railway CEO Jake Cooper
Replit, Doubling down on our commitment to secure vibe coding, July 2025
NHS England, National review of clinical risk management standards DCB0129 and DCB0160: supporting information, 29 June 2026
Santander and Mastercard, joint announcement, 2 March 2026
Salesforce, ServiceNow and Zendesk product documentation on agent capabilities