OpenAI bots thrash team of Dota 2 semi-pros, set eyes on mega-tourney

Next goal: Beat pro players at The International


OpenAI’s machine learning bots have beaten another team of semi-professionals in Dota 2, in their second public match in the traditional five-versus-five settings.

You can watch the action on Twitch – complete with commenters typing in SKYNET! every few seconds – here, or find a summary of the results, here.

The human team – made up of popular Twitch streamers and former professionals ranked in the 99.95th percentile – hunkered down to play against the bots known as OpenAI Five in San Francisco on Sunday. OpenAI Five smashed its opponents, winning comfortably in two out of three games.

It did lose one game, however, after spectators watching the match live and on Twitch were allowed to pick the pool of heroes – the playable characters in the game. Each hero comes with its own strengths and weaknesses and picking a balanced combination is paramount to winning. If you have too many characters for the same role, the other team will steamroll you. The crowd-picked team was imbalanced. It was only under these circumstances that the humans managed to get their own back against the machine.

Dota 2 being played by OpenAI

What does it take for an OpenAI bot to best Dota 2 heroes? 128,000 CPU cores, 256 Nvidia GPUs

READ MORE

Dota 2 is a difficult game to master; it requires teamwork, strategic thinking and lightning-fast reaction times during battles. OpenAI has been working on cracking Dota 2 for a while. Last year, one of its bots won against Dendi, a pro player, under very limited settings that did not represent realistic competitions.

But this year, after cranking up the amount of hardware and engineering required to scale up its machine learning system, OpenAI has lifted a lot of restrictions to level out the playing field between humans and machines.

OpenAI Five is made up five identical long short-term memory networks, each about the size of an ant’s brain, apparently. It learned to play Dota 2 by playing in tonnes of matches against itself, racking up a whopping 180 years’ worth of experience every single day during training. That’s 900 years of collected experience for the five bots – something that human teams can never achieve.

“It has been training pretty non-stop since June and eats up a CPU for every single game,” Jie Tang, a member of OpenAI’s technical staff told The Register.

Round Two

Now, the bots can play in the traditional five-versus-five settings. In the first match in June, OpenAI won comfortably against a team of slightly weaker semi-pros ranked in the 93rd to 99th percentile in mirror matches - where the bot and human teams play with the exact same heros: Necrophos, Sniper, Viper, Crystal Maiden, and Lich.

It was trained on Proximal Policy Optimization, a reinforcement learning algorithm. For the match in June, the system guzzled a mind-boggling 128,000 CPU cores and 256 Nvidia P100 GPUs on Google Cloud.

Tang said that training for the match this time around required even more CPUs and GPUs. The amount of compute needed for training increased from 40 petaflop/s-days for the system used in the first semi-pro match, to 190 petaflop/s-days for the latest practice match.

All that heavy number-crunching is needed as some of the previous restrictions have been lifted. These include:

  • The number of heroes playable in a single match went up from five to 18
  • No mirror matches, both teams can choose freely from the pool of 18 heroes
  • Reaction time has increased from 80 milliseconds to 200 milliseconds
  • No scan – meaning players cannot scan the minimap to scout out unseen local enemies
  • More items were allowed to make the game more complex. For example, items such as Bottle, which restores health points, can now be used by players

But OpenAI Five has still retained some of its major advantages. It can still see the whole map at once every few frames, making it easier to calculate the range of its attacks. Although its reaction time has been increased and is closer to the speed of a human’s, the machines still have the advantage.

Stephen Merity, an ex-Salesforce AI researcher, pointed out that humans can be just as quick at clicking a button, but only when it comes to those moves where it's likely to be a less important and more down to muscle memory. But for the crucial decisions that require more strategy, such as checking the enemy’s inventory and remembering what spells have been cast before making a move during a battle takes longer than 200 milliseconds to execute for humans.

Adding more heroes makes the game much more complex as there is more information and there are possibilities to take into account. In the post-match interview, a panel of OpenAI staff said that it increases the number of different possible game combinations to over 11 million.

It’s impossible to play all these games, but OpenAI Five thrashed the human team anyway. It stuck to close to its strategy of playing aggressively right at the beginning, making a beeline to destroy enemy towers, and killing its opponents.

In the first game, it calculated that its chance of winning was 99 per cent within the first five minutes. And after 21 minutes, the human team lost after the bots took over the map. The humans did slightly better in the second match and held on for 25 minutes.

William Lee – aka “Blitz” – an ex-pro Dota 2 player and coach, said it felt “pretty weird” playing against the bots. “We didn’t know what to expect. It felt like playing against a really solid team. They respond well, and have good teamwork”.

All previous matches have been practice and used to benchmark the machine’s progress in time for its ultimate match at The International, the annual Dota 2 tournament, held in Vancouver, Canada, with the total prize pool of $24m at the end of this month – though the bots won't be able to partake in this.

OpenAI Five will play against a team of professionals under the same restrictions, which limit the full complexity of the game, which has over 100 heroes. Lee was asked whether he expected OpenAI Five to win this time around. After a little hesitation, he said: “I’m going to have some faith in it and say yes, it’ll beat the pro team.” ®


Keep Reading

Twitter: Our image-cropping AI seems to give certain peeps preferential treatment. Solution: Use less AI

Let's just go back to human-selected cropping, eh?

AI in the Enterprise: How can we make analytics and stats sound less scary? Let's call it AI!

Register Debate New names for old recipes

Got a problem with trust in AI? Just add blockchain, Forrester urges. Then bust out the holographic meetings. Welcome to the future

It takes 'grit' to send in a holograph to meetings instead of struggling with mute buttons yourself...

Linux Foundation projects on AI and data merge – because one of these concepts simply can't exist without the other

Open Source Summit Europe Combined org hoping to attract more industry support

AI in the enterprise: AI may as well stand for automatic idiot – but that doesn't mean all machine learning is bad

Register Debate Is AI just a rebrand of yesterday's dumb algorithms? We present the argument against this motion – and don't forget to vote

Microsoft builds image-to-caption AI so that your visually impaired coworkers can truly comprehend your boss's PowerPoint abominations

Better-than-before code to make Office more accessible

Google previews Document AI for parsing forms: Just a catch-up with AWS and Azure?

Convert a jumble of documents into neatly structured data - if you are lucky

UK reveals new 'National Cyber Force', announces Space Command and mysterious AI agency

Combined Ministry of Defence and GCHQ team has worked since April to 'transform cyber capabilities'

Biting the hand that feeds IT © 1998–2020