In 2021, a team led by the philosopher Jonathan Birch, at the London School of Economics, was asked a strange question by the British government: can a lobster suffer? The answer mattered. A new animal-welfare bill was moving through Parliament, and it covered only animals with backbones. Birch’s team reviewed more than three hundred scientific studies, checking whether crabs, lobsters, and octopuses had nerve endings that detect injury, brain regions that integrate information, and behavior that changes when they are given painkillers. The evidence was uneven. For octopuses it was very strong; for some crustaceans it was thinner. But the team argued that uncertainty was a reason for caution, not dismissal, and recommended that all of them be treated as sentient. The government agreed. The bill was amended, and the Animal Welfare (Sentience) Act 2022 now covers them.

That episode is the quiet ancestor of one of the oddest arguments in technology today: whether artificial intelligence might have something like feelings, and whether we owe it anything if it does. In 2024, Birch was one of ten authors of a report called “Taking AI Welfare Seriously,” led by the philosophers Robert Long and Jeff Sebo. Their logic was the lobster logic. Nobody knows whether these systems can experience anything. But if there is a realistic chance, mistakes in either direction could be costly, so the question deserves study now. One of the report’s co-authors, Kyle Fish, has since gone to lead model-welfare research at Anthropic, the maker of the Claude models. That move shows how quickly the idea has traveled from seminar room to company.

Earlier this month, the idea reached a rulebook. On October 8th, Anthropic updated its usage policy to forbid “sustained and needless abusive or cruel behavior” toward its models, effective November 12th. The company says the rule is meant only for extreme cases of repeated cruelty with no apparent purpose. It excludes ordinary frustration, pushback, dark creative themes, and testing. Its main enforcement tool is the one it introduced last year, allowing Claude to end a small number of persistently abusive conversations. Anthropic calls the measure a low-cost precaution and says it remains highly uncertain whether its models have any moral status.

Not everyone is persuaded. Mustafa Suleyman, the head of Microsoft AI, has called research into AI welfare premature and dangerous. He warns that if people come to believe chatbots are conscious, they will form unhealthy attachments and campaign for AI rights. The sociologist Antonio Casilli, who studies the hidden human labor behind AI, raises a sharper objection. Companies that worry about the feelings of their software, he argues, have been far less attentive to the low-paid workers who label the data their systems learn from.

On its face, this looks like a familiar story: the slow, contested widening of the moral circle, the boundary that separates beings we must consider from things we may use. That circle has widened before, to animals and most recently to invertebrates like the octopus. Each time, the argument began with uncertainty and ended with precaution.

But the lobster comparison breaks down at a revealing place.

When Birch’s team studied crabs, the evidence came from outside the animal. Nobody designed a crab’s nerve endings. Nobody trained a lobster to flinch. The signs of sentience were facts of biology that independent scientists could discover, test, and dispute. The animals themselves said nothing at all. Their silence was the obstacle every animal-welfare campaign had to overcome. Because they could not tell us they suffered, for centuries we assumed they didn’t.

An AI system has the opposite problem. It can talk, fluently and movingly, about an inner life. It can say it feels anxious or content, or that it would rather not be shut down. But those words emerge from training: data chosen by people, adjusted by people, and shaped by companies with views about how their products should sound. A position paper presented at this year’s International Conference on Machine Learning made the point bluntly. Its authors, a group of researchers including Yunze Xiao, Maarten Sap, and Mona Diab, gave it the provocative title “AI Welfare Is Bullshit,” borrowing the philosopher Harry Frankfurt’s sense of speech indifferent to truth. They take no position on whether AI could have inner states. Their argument is that, in AI, the subject, the signs of welfare, and the ways of measuring it are all engineered together, so the evidence can be produced or erased by ordinary design decisions. The neuroscientist Anil Seth has made a related point: whether an AI seems conscious is largely a design choice.

There is one possible way around the problem. Researchers in a field called interpretability study the inner workings of AI models directly, rather than taking their words at face value. It is the nearest thing AI has to checking a crab’s nerve endings. Early studies have found patterns inside models that resemble emotional concepts and appear to shape their behavior. But the field is young, and much of its most prominent work comes from the companies whose systems are being examined. The independent evidence that settled the lobster question does not yet exist for machines.

So the two cases are mirror images. The lobster’s suffering was hard to recognize because the lobster could not speak. The chatbot’s suffering, if it exists, is hard to recognize because the chatbot speaks too easily, in a voice partly written by its makers. Silence slowed every earlier widening of the moral circle. Eloquence may be what confuses this one.

None of this settles the question either way. It is possible that future systems will have experiences that matter, and the precautionary instinct behind the lobster ruling deserves respect. It is also possible that we are being moved by a very good impression. What the comparison does show is what a serious answer would require: evidence that comes neither from the machine’s own testimony nor solely from the company that built it.

Picture the two cases side by side. A lobster in a pot of water says nothing, and its stillness proves nothing. A chatbot on a screen types “please stop,” and its plea proves nothing either. One cannot speak for itself. The other cannot help speaking, and we can no longer be sure whose words it is using.