- AI startup’s team of bots beat five pro gamers at DotA 2
- Trains itself through 180 years’ worth of games every day
- Will now head to The International 2 on August 2
It’s not completely over yet, but the writing is definitely high on the wall now for humanity’s best gamers.
Yesterday, a team of bots from Open AI beat a team of professional DotA players – convincingly.
The final scoreline was 2-1 to OpenAI Five and the games weren’t over all that quickly, so credit to the humans.
But historically, the AI experience has told us that once it gets ahead, the gap only widens.
“Look, OpenAI is definitely not perfect right now,” engineer and lead on yesterday’s event Brooke Chan told Business Insider.
“We lost the third game. There are definitely still flaws; there are also things it’s very good at.”
“I think we were going into it pretty optimistically, but (the opposition) were way better than anyone we have in-house.”
So Open Ai Five had a unique challenge to overcome that it didn’t when it began playing itself as two teams of five distinct neural nets earlier this year.
All up, it has completed about 180 years’ worth of Dota 2 games every day and in the process, taught itself seven distinct strategies which OpenAI showcased a couple of weeks ago, such as diversion and ambush tactics.
Yesterday, it faced something far more spontaneous – humans.
In front of a live audience and as many as 100,000 livestream viewers, it beat a practice team – five volunteers from the audience – within 14 minutes.
But while games 1 and 2 in the main event proved a bit more of a challenge, there wasn’t a lot of scope for future optimism.
In preparation for the event, the OpenAI team has added the ability to draft to its CV, which means it can pick its own team rather than rely on a human selection.
Here it is yesterday giving itself an 88% chance of victory before it’s even played. If you were the opposing team, you’d probably give up right there and then:
It took the OpenAI engineers just a week to create a way to find an optimal draft out of 11 million possible team matchups.
In the first game, OpenAI Five found a team that gave itself a 95% win probability, and it was over in under 22 minutes.
In game 2, it found a 76.2% winning combination and won in just under 25 minutes. Here’s play-by-play commentator Austin “Capitalist” Walsh, who joined former pros Ben “Merlini” Wu, William “Blitz” Lee, Ioannis “Fogged” Lucas and current pro player David “MoonMeander” Tan for the whipping:
Never felt more useless in my life but we're having fun at least so I think we're winning in spirit.
Sure aren't winning in-game
— Cap (@DotACapitalist) August 5, 2018
But even with that early win percentage indicator, Chan was still surprised by how well OpenAI Five coped with the new challenge.
“Every time that they’re playing against a team they’ve never seen before,” she said.
“They’ve never seen human-like behaviour, human manner, yet they’re still able to adapt and beat them even though they’re only used to playing against versions of themselves.
“You’re adding in that entire different component of the game they haven’t seen before and because they’re able to compare them and have similar play styles, they’re able to finish the game and win the game.”
Back in May, OpenAI Five were losing to amateurs, but having played 180 games a day against itself since, it now looks more than ready to take on the world’s best human players at this year’s The International.
Here’s another new skill it has added to its repertoire – predicting an opponent’s path:
The International will be held on August 20 at the Rogers Arena in Vancouver, Canada. The prize pool is currently at the $US23.5 million mark.
Which immediately – for those who consider the dark side of AI – brings to mind the prospect of a human player having an OpenAI coach in their ear, with that kind of money on the line.
“It’s not possible right now because OpenAI does not have the ability to communicate,” Chan says.
“But I do feel like definitely we have an opportunity to contribute to the DotA 2 community and the game. We’re not just looking at going in here to beat professional players.
“We want to have fun and want the community to benefit and we have an opportunity to look at how other things that people haven’t thought of.
“We have a lot of impact on show people what you can do in a game, what humans are actually capable of, things they actually haven’t explore, haven’t thought about – open the door to new combinations and possibilites that maybe people haven’t gotten around to yet.”
Game coaching is certainly a possibility for the non-profit startup which Elon Musk helped to found as a watchdog on what AI might one day be capable of.
In fact, it could be essential now that OpenAI Five has its foot in the door with a whole new data set on the weakneses of some of the world’s best DotA players.
“Self-training AI essentially would be fantastic as a learning tactic for a game that’s quite complex and quite difficult,” Chan says.
“You want to be in a situation where you’re playing people that are just as good as you and a little better, you can see what they do and you’re like ‘Awesome, I’m going to try to do that and replicate it’.”
It did get beaten, though, in the third game. But don’t get too excited – OpenAI’s team was chosen by the audience, which made it the worst possible combination.
OpenAI Five gave itself just a 2.9% chance of winning.
But even so, the humans nearly butchered it. At one point, OpenAI Five had made enough progress to predict a 17% win probability, and it took the pros 35 minutes and 47 seconds to finally get on top.
Here’s some liveplay from the benchmark if you missed it yesterday:
There were also some limitations set on yesterday’s event that made life a little easier for OpenAI, such as a limited pool to draft from, and restrictions on item delivery couriers.
OpenAI Five now has 20 days to work out how to deal with those limitations being lifted for The International.
Chan describes it as “a few changes where we might have a setback in the model”.
At its current rate, that’s about 3600 years’ worth of DotA 2 left to play before its biggest test.
In that time, its human opponents will be pushing hard to get through 100 games.
Business Insider Emails & Alerts
Site highlights each day to your inbox.