OpenAI engineer Christy Dennison (left) and gamer William Lee commentate on a Dota match between OpenAI Five and a human team. Via OpenAI.

OpenAI Five ‘Hammers’ Humans in Team-Based Game

The Elon Musk-founded nonprofit OpenAI announced this week that its new AI OpenAI Five soundly defeated amateur teams of human opponents in the game Dota 2, earning the organization kudos from Bill Gates who called the success a “huge milestone” in AI development.

Developed by Valve Corporation, the wildly popular Dota 2—or simply “Dota” in gaming circles—is based on two opposing teams of five players each pitting fantasy-themed bosses against each other on a 3D map. The object of the game, according to professional Dota gamer and esports commentator William Lee (known in Dota circles by nom de guerre “Blitz”), is simple: “kill the enemy team, take the buildings.”

Despite Blitz’ spartan summation of Dota, the game and others like it are extraordinarily complex for AI to master—magnitudes more so than chess or go—due to the sheer number of variables a modern 3D game environment entails, the “imperfect” informational nature of the game and the swift maneuvering and close cooperation between teammates that successful multiplayer strategy requires.

Human Dota players practice for years to master the game and compete in star-studded international tournaments including The International, an annual high-stakes battle royale with purses up to $40 million. The International has become the surprising new frontline for cutting AI development. Last year, OpenAI successfully bested the world’s best Dota player in one-on-one exhibition, and this year OpenAI’s development team seeks to repeat its achievement with the advent of OpenAI Five, which is capable of controlling a full team of five AI players to play in a basic version of Dota 2 with some restrictions on rules.


Above: OpenAI’s first Dota bot defeating a top-ranked human player at The International 2017. 

According to a press release and accompanying video on OpenAI’s blog posted June 25, the research team is making headway towards its goal for Dota supremacy and making AI breakthroughs in the process. OpenAI Five defeated teams of human opponents this spring, facing off against informal teams from OpenAI and Valve employees as well as an audience team with some seasoned players including Blitz.

To the surprise of the team, the AI has also defeated regularly practicing teams of amateur and semi-pro Dota players, defeating the human team in two of the first three games despite OpenAI’s prediction that OpenAI Five would “lose soundly” against coordinated human players.

Above: “Team Human” suffering defeat at the hands of OpenAI’s Dota bot earlier this year. Via OpenAI

Blitz described OpenAI Five’s gameplay ability as “insane” and “intuitive,” praising the AI for its ability to employ advanced strategies and capitalize on errors. “It didn’t mess up,” he said; “it felt like I was just getting hammered every single time that I made a mistake, and I feel like normal humans don’t do that.” According to the player, OpenAI Five’s team strategies demonstrated an ability that human players would take nearly a decade to uncover, saying “it [took] about eight years for me to learn the strategies that the bot was intuitively doing.”

Blitz’ commentary should come as no surprise given OpenAI Five’s training regimen at OpenAI headquarters. Describing the structure of the bot’s machine learning architecture and training methods used on OpenAI, the team’s press release revealed that OpenAI’s strategies were developed by reinforcement learning on 356 GPUs and 128,000 CPUs, playing the equivalent of 180 years of human gameplay every single day.

While AI-dominated Dota gameplay is an impressive achievement, winning an online game in and of itself hardly warrants a mention from Bill Gates; even more profound is the AI’s potential to excel in more ambitious arenas than gaming alone. According to OpenAI machine learning engineer Christy Dennison, “overall, what we’re excited about is that the training method we use is very general. We’re focused on learning Dota, but we’re hoping this will give us more and more insight about how AI can solve complex problems of any kind.”

Above: OpenAI Five’s training architecture. Via OpenAi.

OpenAI’s press release further explores the reasons why Dota success may foretell broader advances in AI. Dota’s strategic complexity and the open, higher-dimensional world of 3D gameplay naturally require more able AI than those used in the comparatively pedestrian 2D planes of chess or go, but the real reason that Dota presents such a computational challenge is its “imperfect” informational state.

Unlike chess, go or other games of “perfect information” where all players can see the entire state of the game, Dota, poker, and real life itself are games of “imperfect” information, meaning that the ground state of other players/adversaries’ positions remains unknown and must be intuited and predicted in order to develop winning strategy.

According to OpenAI, the Dota AI succeeds in an environment more akin to “the messiness and continuous nature of the real world” than board games, an important step towards creating artificial general intelligence (AGI), an advancement OpenAI predicts will become “the most significant technology ever created by humans.” Concerned about the future of AI, Elon Musk co-founded OpenAI in 2015 “to build safe AGI, and ensure AGI’s benefits are as widely and evenly distributed as possible.” The organization, now lead by Sam Altman following Musk’s official departure to avoid conflicts of interest with Tesla Motors, conducts cutting-edge AI research and publishes reports on AI advancements.

OpenAI Five is scheduled to compete against human Dota teams at The International 2018, hosted August 20–25 in Vancouver. Prior to its shot at glory, the AI will also make its public debut during a live-streamed bout against a team of “top players” on July 28.