Given the recent superhuman performances of AI models in Go and complex real-time strategy video games like StarCraft, it would seem the good old Chess board offers no challenge whatsoever for today’s advanced algorithms. But what if an AI could learn to play Chess based only on human game commentary? That interesting task has been tackled by SentiMATE, a new Chess algorithm proposed by University College London.
Researchers extracted Chess commentary texts describing the quality of moves, then used sentiment analysis to capture the underlying emotions of the commentators and teach the model to play Chess.
SentiMATE is an end-to-end deep learning model that employs natural language processing (NLP) to perform an evaluation function assessing Chess move quality. This pretrained sentiment evaluation function is used to optimize the AI’s decision-making process in actual gameplay.
Unlike other Chess engines, SentiMATE does not require extensive manual feature data or large amounts of specific domain knowledge. The research team used a relatively small database of 2,700 annotated Chess game commentaries available online, including information on the various moves’ strategic value along with commentators’ discussion and assessment of the moves.
Researchers used Flair, an NLP model developed by Zalando Research, to split the commentaries into two classifications: Quality and Non-Quality. Flair combines long short-term memory (LSTM) and appropriate word embeddings to effectively learn word usage regarding Chess games. With a structured dataset containing both text-based commentary and state representation (before and after the move has occurred), researchers formatted the state representations into the evaluation function, as shown in Figure 1 below.
The achievements of SentiMATE are promising. The model beat both a random strategy agent and an implementation of DeepChess, the first end-to-end machine learning-based model for Chess. In 100 games playing as both black and white, SentiMATE achieved an 81 percent win rate against the random agent and defeated DeepChess on search depth 1 (indicating the number of moves ahead the agent will explore).
SentiMATE’s novel Sentiment Analysis model based on Chess commentary introduces a new method for NLP-based move evaluation. Researchers believe the method has the potential for practical applications beyond Chess, for example sentiment analysis could also enable machines to use emotional content in tasks such as financial activity prediction, sports commentary, and recommendations.
The paper SentiMATE: Learning to play Chess through Natural Language Processing is on arXiv.
Author: Hongxi Li | Editor: Michael Sarazen
Pingback: AI Analyzes Chess Commentary to Learn to Play Chess – Synced – IAM Network