Ready, Set, Go! Meet Yuandong Tian: AI Maverick

Chinese AI talent in US – Facebook Artificial Intelligence Researcher Yuandong Tian

Yuandong Tian loves to seize a challenge. When the ambitious young AI researcher joined Facebook in 2015, he came up with the “crazy idea” of building a Go program from scratch. Go is the world’s oldest and most complex strategic board game, and had long confounded computer programmers. Tian had scarcely played the game. But that didn’t stop him.

Ten months later, Tian took his program, Darkfmcts3, to Tokyo for the 2016 Computer Go UEC Cup, the world’s top Go AI competition. Darkfmcts3 went up against well-known and state-of-the-art Go AIs such as Zen, DolBaram and Crazy Stone — and surprisingly took second place.

Suddenly, Tian and his Darkfmcts3 were in the media and social network spotlight. MIT Tech Review described Darkfmcts3 as a “game-changing Go engine” that successfully applied Artificial Intelligence (AI) to achieve incredible results, while Wired marvelled that Facebook was now challenging the best in the Computer Go race.

Alas, Tian’s underdog was soon overshadowed by Google DeepMind’s powerful AlphaGo — which beat Go Grandmaster Lee Sedol in March 2016. Darkfmcts3, along with all other Go AIs, were left behind.

“I was lucky to be a part of history,” says Tian. Facebook opensourced Darkfmcts3’s code in June 2016. Even today, students are still building computer Go programs based on Darkfmcts3.

AI is the future.

Tian’s work on a Computer Go program was a crazy idea because it had nothing to do with his main research interest: computer vision, which seeks to automate machines to perform like the human visual system. And Go is not the only frontier Tian has explored.

Born in Shanghai, China, Tian received a Bachelor’s and Master’s degree from the elite Shanghai Jiao Tong University’s Computer Science and Engineering Department. In 2008, he was admitted to the Robotics Institute of Carnegie Mellon University (CMU). His renowned graduate paper – which explores how a hierarchical framework gives globally optimal guarantees for non-convex non-rigid image deformation – helped him win the ICCV 2013 Marr Prize Honorable Mention, one of the top honours for a computer vision researcher.

Meanwhile, Tian was watching the rise of deep learning, especially after AlexNet won the 2012 ILSVRC (ImageNet Large-Scale Visual Recognition Challenge, aka the Olympics of computer vision). AlexNet’s performance in image recognition was so revolutionary that Tian started wondering whether he should switch his research approach to deep learning.

Traditionally, a computer vision specialist undertakes image detection by creating a feature descriptor that finds the connection between pixels contained in a digital image and the definition of an image, and then puts features in discriminative classifiers that map input data to a category. But deep learning is a totally different method. Back in 2013, it was unknown if deep learning could be implemented in computer vision. “It was courageous for researchers to shift from traditional approaches to deep learning,” said Tian.

And so Tian decided to start fresh by studying deep learning. He left Google’s Self-driving Car Team for a better fit in Facebook’s AI Research Lab. He surprised even himself when his first Facebook project took aim at Computer Go.

Building a Computer Go program from scratch

Even though IBM’s chess-playing computer Deep Blue had bettered World Chess Champion Garry Kasparov in their famous 1997 showdown, researchers believed machine supremacy at Go remained a long way off. The 2,500 year-old Chinese game is recognized as the most complex strategy board game ever — its 19×19 playing grid has 10^170 legal positions, outnumbering the particles in the observable universe.

Back in 2015, most Go programs were based on the Monte Carlo Tree Search, but the heuristic search algorithm for decision processes still struggled with Go’s challenges. The best Go programs at the time — Zen from Japan and CrazyStone from France — were no better than 6 dan, an advanced amateur level.

Scientists started to apply deep learning to Computer Go, but the best technology of the time managed only 1k-2k, an intermediate amateur level. “There seemed to be a huge space for improvement,” recalls Tian.

In May 2015, Tian asked his Facebook intern Yan Zhu — a PhD student at Rutgers University — if he had interest in building a Computer Go program. “We hit it off immediately,” recalls Zhu.

Tian and Zhu, an amateur Go player, held different opinions on how a researcher’s Go ability would affect their research. Zhu thought a beginner would be unable to evaluate the problem as effectively as professional players, while Tian believed Computer Go should rely on data input instead of human skill at the game.

The pair’s next step was to use deep learning algorithms to train a policy network — an artificial net that could narrow down possible moves based on data. This quickly produced results, and their program was able to reach a level of 1K-2K.

In August, Zhu ended his internship at Facebook and Tian continued on his own. He refined the model in two weeks and put his first Go AI, DarkForest, on the Go online platform KGS to compete against human players. There were bugs: the program was poor at dealing with strategic plays like the Ladder, a basic sequence of moves in which an attacker pursues a group across the board in a zig-zag pattern; and Ko, a repeating series of threats and captures.

Tian improved his algorithms and made a few optimizations. The results were incredible, and as DarkForest boosted its win rate, Tian’s paper on the subject drew attention. Facebook Chairman and CEO Mark Zuckerberg was delighted: “The researcher who works on [Go AI], Yuandong Tian, sits about 20 feet from my desk. I love having our AI team near me so I can learn from what they’re working on.”

To boost Darkforest’s capability, Tian incorporated the Monte-Carlo Tree Search, while an ex-Facebook engineer helped him enable search algorithms by employing many GPUs on many machines. In 2016 Tian’s upgraded version, Darkfmcts3, reached a stable 5 dan level — that of an advanced amateur human player.

Three months after DarkForest was released, Google’s AlphaGo took the spotlight with a significant victory over European Champion Hui Fan, and later beat Korean Go legend Lee Sedol 4-1 in a historic showdown in Seoul.

The DeepMind team included many professional Computer Go researchers. Aja Huang, their senior research scientist, studied how to apply the Monte Carlo Tree Search to Go computers in his PhD dissertation. Another key player, AlphaGo’s lead researcher David Silver, studied for a PhD on reinforcement learning at University of Alberta, where he co-introduced the algorithms used in the first master-level Go programs for a 9×9 board size.

“AlphaGo reversed the long-standing belief that a computer could not beat top humans at Go,” said Tian. “It means a lot to the future. Artificial intelligence became a buzzword because of AlphaGo.”

A talented writer

While AlphaGo was playing Lee Sedol, Tian wrote on Zhihu (China’s Quora), explaining how AlphaGo was able to defeat Go masters. His YuanDongYiShi (远东轶事) blog went viral, and has attracted over ten thousand followers. “It is the best analysis of AlphaGo I have ever seen,” posted Zhihu user “iamsmile.”

Tian loves writing, which is an unusual hobby for a scientific researcher. Back in high school, Tian used his spare time to write articles and novels, most of which were scientific. “Scientists need imagination,” says Tian. One of his proudest productions was a 300,000 word scientific novel that tells the story of four young adults and their voyage on an artificial planet.

“It won’t help the scientific research directly,” says Tian, “but writing novels can improve communication, which I realize is critical to understanding what others are thinking, and opening up the mind.”

Tian says his next goal is to write a novel about AI — which may take some time due to his heavy workload at Facebook. So he writes blogs, sharing life experiences such as his feelings after his first half-marathon, how he manages his schedule, and his tips for researchers. He wakes each morning at 6 or 7 a.m., eats healthy food and works out for an hour every two days. He lives more like a bodybuilder than a programmer.

“Why do neural networks work so well?”

Inspired by AlphaGo, Tian became fascinated with reinforcement learning, an area of machine learning that maximizes output without relying on large datasets. One of his papers last year proposed a new framework for training vision-based agents for First-Person Shooter (FPS) Games by combining reinforcement learning and curriculum learning. Tian and his intern Yuxin Wu developed an agent that successfully won the Track1 championship at the ViZDoom AI Competition 2016 by a large margin, scoring 35% higher than second place.

While Tian put the majority of his time and energy into the application of deep learning and reinforcement learning, his interest in theory continued in parallel. Since he started studying deep learning, Tian has endeavoured to figure out why neural networks work so well, and that’s where his current research is taking him.

“Despite the empirical success of deep learning in Computer Vision, Natural Language Processing and Speech Recognition, it remains elusive on how and why simple methods like gradient descent can solve the complicated non-convex optimization in its training processing,” wrote Tian in the introduction of his latest paper, which studies the gradient descent dynamics of a 2-layered bias-free ReLU network.

Almost all machine learning development requires solving non-convex optimization problems, and this has become Tian’s new focus and long-term goal. “Maybe I might spend my whole life for nothing, or maybe I’ll get the answer suddenly in the bath — who knows?” Certainly this new adventure is more ambitious than building a Computer Go program, but Tian is always up for a challenge. “Scientists need passion to do research, and that is my passion.”

---

Author: Tony Peng | Editor: Michael Sarazen