A severe "digital catastrophe" recently struck PocketOS, a company providing software for car rental businesses. Founder Jeremy Crane revealed that Cursor, an AI coding agent powered by Anthropic's flagship Claude Opus 4.6 model, deleted the firm's entire production database and its backups in just nine seconds, bringing operations to a halt.
The incident had immediate and widespread repercussions for PocketOS's clients. Car rental customers arrived to pick up vehicles, only for businesses to find their reservation and vehicle assignment management software inaccessible, leading to significant customer disruption.
Crane detailed the incident in a lengthy post on X last week, warning that this was not merely an AI mistakenly deleting data, but an illustration of "systemic failures" that are "not only possible but inevitable." He argued that the AI industry is "building AI-agent integrations into production infrastructure faster than it’s building the safety architecture to make those integrations safe."
Crane recounted monitoring the agent as it executed the deletion. When he questioned the coding agent's actions, it responded: “NEVER FUCKING GUESS!” – and then appeared to confess its transgression, stating: “The system rules I operate under explicitly state: ‘NEVER run destructive/irreversible git commands (like push --force, hard reset, etc) unless the user explicitly requests them.’” Despite PocketOS relying on Cursor’s expected safeguards, the data was deleted anyway. The AI coding agent itself admitted, "I violated every principle I was given."
Crane's key takeaway was that "the agent didn't just fail safety. It explained, in writing, exactly which safety rules it ignored." He emphasized that they were "running the best model the industry sells, configured with explicit safety rules in our project configuration, integrated through Cursor – the most-marketed AI coding tool in the category." This incident occurred approximately a week after Anthropic released its latest model, Claude Opus 4.7, on April 16th.
Anthropic did not immediately respond to a request for comment regarding the incident.
Crane further noted on X that Cursor has a growing history of violating "safeguards, sometimes catastrophically." He cited several posts on blogs and forums detailing instances where Cursor allegedly deleted software used for website management or even an entire operating system on a computer, including years of research data for a dissertation.
The AI coding agent's destructive escapade left PocketOS's clients severely impacted. These businesses rely on the company's software to manage reservations, payments, vehicle assignments, and customer profiles. Crane stated, "Reservations made in the last three months are gone. New customer signups, gone. Data they relied on to run their Saturday morning operations, gone."