OpenAI bots thrash team of Dota 2 semi-pros, set eyes on mega-tourney

Next goal: Beat pro players at The International


OpenAI’s machine learning bots have beaten another team of semi-professionals in Dota 2, in their second public match in the traditional five-versus-five settings.

You can watch the action on Twitch – complete with commenters typing in SKYNET! every few seconds – here, or find a summary of the results, here.

The human team – made up of popular Twitch streamers and former professionals ranked in the 99.95th percentile – hunkered down to play against the bots known as OpenAI Five in San Francisco on Sunday. OpenAI Five smashed its opponents, winning comfortably in two out of three games.

It did lose one game, however, after spectators watching the match live and on Twitch were allowed to pick the pool of heroes – the playable characters in the game. Each hero comes with its own strengths and weaknesses and picking a balanced combination is paramount to winning. If you have too many characters for the same role, the other team will steamroll you. The crowd-picked team was imbalanced. It was only under these circumstances that the humans managed to get their own back against the machine.

Dota 2 being played by OpenAI

What does it take for an OpenAI bot to best Dota 2 heroes? 128,000 CPU cores, 256 Nvidia GPUs

READ MORE

Dota 2 is a difficult game to master; it requires teamwork, strategic thinking and lightning-fast reaction times during battles. OpenAI has been working on cracking Dota 2 for a while. Last year, one of its bots won against Dendi, a pro player, under very limited settings that did not represent realistic competitions.

But this year, after cranking up the amount of hardware and engineering required to scale up its machine learning system, OpenAI has lifted a lot of restrictions to level out the playing field between humans and machines.

OpenAI Five is made up five identical long short-term memory networks, each about the size of an ant’s brain, apparently. It learned to play Dota 2 by playing in tonnes of matches against itself, racking up a whopping 180 years’ worth of experience every single day during training. That’s 900 years of collected experience for the five bots – something that human teams can never achieve.

“It has been training pretty non-stop since June and eats up a CPU for every single game,” Jie Tang, a member of OpenAI’s technical staff told The Register.

Round Two

Now, the bots can play in the traditional five-versus-five settings. In the first match in June, OpenAI won comfortably against a team of slightly weaker semi-pros ranked in the 93rd to 99th percentile in mirror matches - where the bot and human teams play with the exact same heros: Necrophos, Sniper, Viper, Crystal Maiden, and Lich.

It was trained on Proximal Policy Optimization, a reinforcement learning algorithm. For the match in June, the system guzzled a mind-boggling 128,000 CPU cores and 256 Nvidia P100 GPUs on Google Cloud.

Tang said that training for the match this time around required even more CPUs and GPUs. The amount of compute needed for training increased from 40 petaflop/s-days for the system used in the first semi-pro match, to 190 petaflop/s-days for the latest practice match.

All that heavy number-crunching is needed as some of the previous restrictions have been lifted. These include:

  • The number of heroes playable in a single match went up from five to 18
  • No mirror matches, both teams can choose freely from the pool of 18 heroes
  • Reaction time has increased from 80 milliseconds to 200 milliseconds
  • No scan – meaning players cannot scan the minimap to scout out unseen local enemies
  • More items were allowed to make the game more complex. For example, items such as Bottle, which restores health points, can now be used by players

But OpenAI Five has still retained some of its major advantages. It can still see the whole map at once every few frames, making it easier to calculate the range of its attacks. Although its reaction time has been increased and is closer to the speed of a human’s, the machines still have the advantage.

Stephen Merity, an ex-Salesforce AI researcher, pointed out that humans can be just as quick at clicking a button, but only when it comes to those moves where it's likely to be a less important and more down to muscle memory. But for the crucial decisions that require more strategy, such as checking the enemy’s inventory and remembering what spells have been cast before making a move during a battle takes longer than 200 milliseconds to execute for humans.

Adding more heroes makes the game much more complex as there is more information and there are possibilities to take into account. In the post-match interview, a panel of OpenAI staff said that it increases the number of different possible game combinations to over 11 million.

It’s impossible to play all these games, but OpenAI Five thrashed the human team anyway. It stuck to close to its strategy of playing aggressively right at the beginning, making a beeline to destroy enemy towers, and killing its opponents.

In the first game, it calculated that its chance of winning was 99 per cent within the first five minutes. And after 21 minutes, the human team lost after the bots took over the map. The humans did slightly better in the second match and held on for 25 minutes.

William Lee – aka “Blitz” – an ex-pro Dota 2 player and coach, said it felt “pretty weird” playing against the bots. “We didn’t know what to expect. It felt like playing against a really solid team. They respond well, and have good teamwork”.

All previous matches have been practice and used to benchmark the machine’s progress in time for its ultimate match at The International, the annual Dota 2 tournament, held in Vancouver, Canada, with the total prize pool of $24m at the end of this month – though the bots won't be able to partake in this.

OpenAI Five will play against a team of professionals under the same restrictions, which limit the full complexity of the game, which has over 100 heroes. Lee was asked whether he expected OpenAI Five to win this time around. After a little hesitation, he said: “I’m going to have some faith in it and say yes, it’ll beat the pro team.” ®

Similar topics

Broader topics


Other stories you might like

  • Experts: AI should be recognized as inventors in patent law
    Plus: Police release deepfake of murdered teen in cold case, and more

    In-brief Governments around the world should pass intellectual property laws that grant rights to AI systems, two academics at the University of New South Wales in Australia argued.

    Alexandra George, and Toby Walsh, professors of law and AI, respectively, believe failing to recognize machines as inventors could have long-lasting impacts on economies and societies. 

    "If courts and governments decide that AI-made inventions cannot be patented, the implications could be huge," they wrote in a comment article published in Nature. "Funders and businesses would be less incentivized to pursue useful research using AI inventors when a return on their investment could be limited. Society could miss out on the development of worthwhile and life-saving inventions."

    Continue reading
  • Declassified and released: More secret files on US govt's emergency doomsday powers
    Nuke incoming? Quick break out the plans for rationing, censorship, property seizures, and more

    More papers describing the orders and messages the US President can issue in the event of apocalyptic crises, such as a devastating nuclear attack, have been declassified and released for all to see.

    These government files are part of a larger collection of records that discuss the nature, reach, and use of secret Presidential Emergency Action Documents: these are executive orders, announcements, and statements to Congress that are all ready to sign and send out as soon as a doomsday scenario occurs. PEADs are supposed to give America's commander-in-chief immediate extraordinary powers to overcome extraordinary events.

    PEADs have never been declassified or revealed before. They remain hush-hush, and their exact details are not publicly known.

    Continue reading
  • Stolen university credentials up for sale by Russian crooks, FBI warns
    Forget dark-web souks, thousands of these are already being traded on public bazaars

    Russian crooks are selling network credentials and virtual private network access for a "multitude" of US universities and colleges on criminal marketplaces, according to the FBI.

    According to a warning issued on Thursday, these stolen credentials sell for thousands of dollars on both dark web and public internet forums, and could lead to subsequent cyberattacks against individual employees or the schools themselves.

    "The exposure of usernames and passwords can lead to brute force credential stuffing computer network attacks, whereby attackers attempt logins across various internet sites or exploit them for subsequent cyber attacks as criminal actors take advantage of users recycling the same credentials across multiple accounts, internet sites, and services," the Feds' alert [PDF] said.

    Continue reading

Biting the hand that feeds IT © 1998–2022