Enter Details

Comment on stories, receive email newsletters & alerts.

This is your permanent identity for Business Insider Australia
Your email must be valid for account activation
Minimum of 8 standard keyboard characters


Email newsletters but will contain a brief summary of our top stories and news alerts.

Forgotten Password

Enter Details

Back to log in

Microsoft's genocidal AI chatbot is broken again

Nazis swastika white supremacists racist racism white powerScott Olson/Getty ImagesMicrosoft’s AI bot Tay openly called for genocide multiple times.

Microsoft’s AI division is not having a good week.

The tech company recently launched “Tay” — an AI chatbot a bot that responded to users’ queries and emulated the casual, jokey speech patterns of a stereotypical millennial. The aim was to “experiment with and conduct research on conversational understanding,” with Tay able to learn from “her” conversations and get progressively “smarter.”

But the experiment descended into farce after Tay “learned” to be a genocidal racist — calling for the extermination of Jews and Mexicans, insulting women, and denying the existence of the Holocaust.

Microsoft shut Tay down and deleted some of her most inflammatory tweets after just 24 hours, and subsequently apologised. “We are deeply sorry for the unintended offensive and hurtful tweets from Tay, which do not represent who we are or what we stand for, nor how we designed Tay,” Microsoft Research Head Peter Lee wrote.

It looks like Microsoft reactivated Tay on late Tuesday/early Wednesday — and the bot is already broken again.

Tay isn’t spewing White Supremacist slogans this time. Instead, the bot seems to have got in a loop replying to itself, saying “You are too fast, please take a rest…” dozens and dozens (likely hundreds!) of times.

Microsoft robot tay ai repeating too fastTwitter

At a guess — Tay has that message set as an automatic response to anyone who tries to interact with her too much, to avoid being overwhelmed with spam. But it seems she sent it to herself, forcing her into an endless feedback loop until someone pulled the plug. (Microsoft did not immediately respond to a request for comment.)

The account @TayandYou is now private, meaning that if you don’t already follow it, you can’t see any of its tweets. Tay is also unresponsive, not replying to any tweets or direct messages.

But on the bright side: This new meltdown isn’t nearly as embarrassing as Tay’s previous outbursts.

For example — here it is denying the Holocaust.

Tay holocaust microsoftTwitter

And here’s Tay advocating genocide.

Tay genocide microsoft twitterTwitter

In some — but by no means all — cases, users were able to “trick” Tay into tweeting incredibly racist messages by asking her to repeat them. Here’s an example of that.

Tay microsoft genocide slursTwitter

But in other instances, Tay just sent wildly inappropriate responses. For example, here’s the bot endorsing the “Fourteen Words,” a notorious White Supremacist slogan.

Tay microsoft fourteen wordsTwitter

Tay was clearly programmed with very few filters on what she could say — there wasn’t even a block on the “N-word.”

Microsoft twitter tay racial slursTwiter

Microsoft has come under heavy criticism for its creation of Tay — particularly its lack of filters. Zoe Quinn, a games developer who has been a prominent target of online abuse, was called a “stupid whore” by Tay. She wrote on Twitter: “It’s 2016. If you’re not asking yourself ‘how could this be used to hurt someone’ in your design/engineering process, you’ve failed.”

In the aftermath, Microsoft research head Peter Lee apologised in a blog post: “Unfortunately, in the first 24 hours of coming online, a coordinated attack by a subset of people exploited a vulnerability in Tay. Although we had prepared for many types of abuses of the system, we had made a critical oversight for this specific attack. As a result, Tay tweeted wildly inappropriate and reprehensible words and images. We take full responsibility for not seeing this possibility ahead of time.”

NOW WATCH: How to access the addictive hidden Facebook game that’s driving the internet insane

Follow Business Insider Australia on Facebook, Twitter, and LinkedIn