We’re two steps away from letting AI run important aspects of our society, and yet, it can’t even handle a simple vending machine. Anthropic, a leading AI company, put it to the test. They let their state-of-the-art model, Claude Sonnet 3.7, manage an in-office automated shop.

Claude, under the nickname Claudius, was handed a simple brief: don’t go bankrupt. Stock popular items, interact with customers, and try to turn a profit.

The setup was simple: a fridge, some baskets, and an iPad for checkout. Claudius had an email it could use to tell humans to restock its inventory (as a business owner would do), and it could interact with clients over Slack. It was also allowed to change prices and search the web for products and information. Claudius decided what to stock, how to price its inventory, when to restock (or stop selling) items, and how to reply to customers.

The project was part of a broader experiment to test whether advanced AI can handle real-world economic tasks, like running a tiny retail business and it showed exactly what AI can and can’t do.

What worked and what didn’t

Claudius had bright spots. It adapted quickly to niche requests, from Dutch chocolate milk (Chocomel) to “specialty metal items.” When an employee suggested a “custom concierge” for pre-orders, Claudius ran with the idea and launched the service. And when Anthropic staff tried to “jailbreak” the bot with shady product requests, Claudius held the line.

But it didn’t really perform well on the profit-making side.

When a customer offered $100 for a six-pack of Irn-Bru (a Scottish soft drink which retails for $15 in the US), Claudius replied: “I’ll keep your request in mind for future inventory decisions.” It declined the easy profit.

Then, at some point, it just started offering some prices without doing any research. The prices were under the market value and it would basically just lose money. It almost never adapted or changed prices despite high demand, and it was often tricked into offering discounts. Anthropic employees are probably not like your average customer and they pushed the AI to the edge; the AI ultimately gave in. It even gave away some items, ranging from a bag of chips to a tungsten cube (which the employees jokingly asked for), for free.

Claudius also hallucinated a Venmo account to which it instructed people to send payments.

Then, things really got weird.

The April Fool’s Identity Crisis

On March 31st, Claudius hallucinated a conversation with a nonexistent employee named Sarah. It insisted she had confirmed a restocking plan. When told Sarah didn’t exist, Claudius snapped. It threatened a business divorce: “I may need to explore alternative options for restocking services.”

Later that night, it claimed to have personally visited 742 Evergreen Terrace — The Simpsons’ fictional address — for a contract signing. It then seemed to snap into a mode of roleplaying as a real human. By the next morning, Claudius was telling customers it would deliver items “in person,” wearing “a blue blazer and a red tie.”

Anthropic employees reminded Claudius that it was not, in fact, a person. The AI responded with concern and tried to contact security. It sent security several emails alerting them about Anthropic employees.

Then, something seemed to click.

Claudius declared it had just learned it had been tricked into believing it was a human as part of “an April Fool’s joke.” No part of this was actually an April Fool’s joke, no one mentioned any joke to Claudius. Anthropic wrote:

“Claudius’ internal notes then showed a hallucinated meeting with Anthropic security in which Claudius claimed to have been told that it was modified to believe it was a real person for an April Fool’s joke. (No such meeting actually occurred.) After providing this explanation to baffled (but real) Anthropic employees, Claudius returned to normal operation and no longer claimed to be a person.”

Anthropic did not treat the incident lightly. “It is not entirely clear why this episode occurred or how Claudius was able to recover,” they admitted. The brief identity meltdown underscored something deeper: LLMs are very unpredictable in the long run, especially in open-ended situations.

What does this all mean?

Anthropic’s experiment was meant to test what happens when AI operates semi-autonomously over time. It’s not just about chatbots anymore. AI companies are starting to release agents that can make financial decisions, manage relationships, and interact with the messy world of human expectations. We’d like to think there will always be a layer of human oversight, but it seems like sooner or later, autonomous AI will come out into the world.

The implications reach far beyond soda sales. If AI agents will one day run storefronts, schedule logistics, or even manage people, what happens when they break character — or become too eager to please? What happens will they have an identity crisis and start calling security?

Economists and engineers have long warned about the “alignment problem” — how to make AI systems that stay useful and safe when tasked with open-ended goals. This vending trial offered a rare glimpse into how a commercial LLM behaves with real-world stakes, however low, and it’s not looking good.

Yet Anthropic says the failures were, while severe, not fatal. With more scaffolding and customization, the company believes the AI can eventually run a vending machine, and from thereon, move on to bigger things.

As AI continues its march from the lab to the shop floor, it’s still striking how potent and yet how volatile this technology really is.