It started with chess. It moved on to Go. Now, AI is ruining computer games for us after beating humans in Dota2.

Screenshot from Dota2.

There’s a distinct difference between games like chess and Go as compared to most strategy computer games: vision. In the first case, the board is open and visible to everyone. But in computer games, you often have what is called the fog of war — enemy units, and often terrain, are hidden from the player unless directly explored. For Artificial Intelligence (AI), dealing with this type of uncertainty is incredibly problematic and difficult to manage. Chess and Go are also turn-based games, whereas in Dota2, the computer needs to react and adapt in real-time.

[panel style=”panel-default” title=”Dota2″ footer=””]Dota2 (originally: Defense of The Ancients) is a free-to-play team game played 5v5. Each team occupies a base on one side of the map, and the purpose of the game is to destroy the other base. Each of the ten players independently controls a powerful character, known as a “hero”, who all have unique abilities and different styles of play. During a match, players collect experience points and items for their heroes to successfully battle the opposing team’s heroes in player versus player combat. A game typically lasts about 30-60 minutes. [/panel]

To teach AIs to play the game, OpenAI, a nonprofit AI research company co-founded by Elon Musk, used a technique called reinforcement learning. Essentially, the AI is given the basic capability to play the game and then is left to its own devices. It plays more and more, learning from its mistakes and improving iteration after iteration. The programmers set different reward criteria that it tries to optimize this trial-and-error approach, but there’s no shortcut — it needs to play a lot of games.

To reach its current level, the AI had to play 180-years’ worth of games every day for 19 days. Then, it was faced against very skilled amateurs (ranking top 1% in the game) — and it beat them. Of course, there’s still a way to go before the AI can square off against the best of the best, but beating skilled players is extremely impressive, particularly considering the sheer amount of chaos and hidden information in the game. It also works as a proof of concept, showing that the AI can improve in a reasonable amount of time, and there’s no reason why it couldn’t progressively improve until it masters the game and becomes unbeatable.

We’ll see if this is the case a bit later. The International 2018, Dota’s flagship tournament, is set to kick off in August, when an exhibition match will be held between leading pro players and these AIs. I can’t wait to see what happens.