With board games like Go already mastered, researchers are now testing their cutting-edge AI techniques in the domain of multiplayer online battle arena (MOBA) games, which involve complex state and action spaces and advanced real-time prediction and decision-making capabilities.
OpenAI has been working on the game Dota 2for three years. Tencent AI Lab meanwhile took up the challenge on Wangzhe Rongyao, a popular MOBA game developed and published by its own TiMi Studio.
Roughly translated as “Honor of Kings” or “King of Glory,” Wangzhe Rongyao was launched in 2015 exclusively for the Chinese market. It reached 80 million daily active players and 200 million monthly active players and became the world’s highest-grossing game across platforms in March 2017. Months later, its international version was released as “Arena of Valor.”
In a recent paper Tencent AI Lab researchers present a deep reinforcement learning (DRL) approach to handle the complex action control of agents in MOBA 1v1 games. The DRL framework is behind Tencent’s Juewu, the AI agent that debuted in August during China Joy — a digital entertainment expo held annually in Shanghai — and achieved a 99.8 percent win rate in over 2,100 games against top human players.
Due to its low coupling and high scalability, their system enables efficient explorations at large scale, the coauthors wrote. They designed a scalable and loosely-coupled system architecture consisting of four modules — AI Server, Dispatch Module, Memory Pool, and RL Learner.
The AI Server implements how the model interacts with the environment. The Dispatch Module is a station for sample collection, compression and transmission. The Memory Pool is the data storage module, which provides training instances for the RL Learner. The paper suggests that such a system design is also applicable to other multi-agent competitive problems.
Tencent AI Lab told Synced that the framework and algorithm proposed in the paper will be open-sourced in order to promote further research on complex simulation scenarios. Tencent also plans to provide Wangzhe Rongyao’s game environment and related computing resources as a virtual cloud for future AI model training.
The company believes the research on AI agents in MOBA games will in the long run advance the ultimate goal towards artificial general intelligence (AGI) development.
The paper Mastering Complex Control in MOBA Games with Deep Reinforcement Learning has been accepted by AAAI 2020 and is available on arXiv.
Journalist: Yuan Yuan | Editor: Michael Sarazen
Pingback: Tencent AI 'Juewu' Beats Top MOBA Gamers | First Option