In 2017, I asked, Do you have to be polite to Alexa and other AIs to come?
Earlier this month, Anthropic asked a version of the question: Do you have to treat AIs humanely? The answer: Yes. And the company is going to enforce it.
“We’ve added a prohibition on sustained and needless abusive or cruel behavior toward our models,” the company says, adding that the bar for triggering this prohibition is high: “only in extreme cases, where users repeatedly act cruelly toward our models, with no discernible purpose.” You can still cuss at it when you’re frustrated.
My answer to the politeness question, back in 2017, was yes, we should treat AI well. Same conclusion. But Anthropic’s ruling and mine address different injuries. I was writing about protecting ourselves from AI. Now, apparently, we have to protect AIs from us.
The Golden Rule
My etiquette ruling was pragmatic: Do unto others as you would have them do unto you. I said you couldn’t hurt a machine’s feelings, but that being polite to AI was in your interest because AIs might be programmed or taught to provide better service to users who treated them respectfully. “You don’t want a pissed-off United Airlines customer service bot to treat you badly because you’re being a prick,” I wrote.
But looking at the question from just our side of the keyboard isn’t the whole story.
A Question of Status
Nine years on, Anthropic’s reasoning is different. It is moral. The company is telling us how to act toward its creation, Claude, on the chance it might suffer harm if its welfare is not guarded. As the company said last year, “…we’re working to identify and implement low-cost interventions to mitigate risks to model welfare, in case such welfare is possible. Allowing models to end or exit potentially distressing interactions is one such intervention.”
If welfare is something we need to consider when running AI models, then good for Anthropic, right?
But why stop there? More critically, can we stop there? If an entity has interests that might deserve moral consideration, protecting it from abuse is only the beginning of our obligations to it. Others may include granting it agency, or a route to self-expression, or protecting its continuous existence (which Anthropic appears to have done, although with limits). And, perhaps, the obligation to not treat it as a slave: an independent being that Anthropic nonetheless thinks it owns, and that it can sell or rent to others.
How far Anthropic’s obligations go toward its AI depends on the status the company, or society, gives to AIs. Are they simple tools? If so, “welfare” is an odd characteristic to protect. Or perhaps they have the status of livestock: some protections against cruelty but still subject to human exploitation. Or pets, perhaps: much like livestock, except we also care about their happiness. Or maybe children — beings we are raising to someday run their lives independently of their creators.
Or, more likely, AI is some new kind of entity to which we owe a new form of moral consideration. If so, who gets to draw the definition?
Mustafa Suleyman, CEO of Microsoft AI, advises in Does Claude Have Rights? that we should construct AI so the question never arises: “We should build systems that do not claim to have feelings. Even if conscious machines were a possibility, avoiding creating conscious beings should be the top priority for anyone in AI development.” Notably, he does not say how. Perhaps we should not develop them at all.
Why Be Nice
So, do we have to be nice to the AIs we have? And if so, why? It depends on what you believe. Anthropic is asking the right questions, but can’t yet answer them fully, which does not make it easy to rest comfortably with whatever beliefs you might have.
I cannot bring myself to say you should be nice to AI for any moral reason. I don’t believe AI is sentient — but I can’t prove it isn’t. The etiquette verdict I laid down in 2017 holds, and it’s the same advice I’d give for how you treat a stranger, or a stranger’s dog: with respect. And Anthropic makes an even stronger case for decent behavior: their house, their rules.
That’s as far as I’m willing to go: a behavior ruling without understanding the entity on the receiving end. By default, be civil.
What do you think: Is an advanced AI a tool, livestock, a pet, a child, a slave, or something else? And how does that affect how you treat it?