DeepMind, which created the epoch-making AlphaGo, has once again made a remarkable achievement in machine learning. Researchers from the British AI company yesterday published a paper on Nature — Vector-based navigation using grid-like representations in artificial agents — which proposes an artificial virtual agent that can navigate like mammals.
Humans and other animals navigate from one location to another seemingly without thinking, bypassing obstacles and even finding shortcuts. Scientists have identified three kinds of brain cells related to navigation ability: Place Cells memorize past locations, Head-Direction Cells sense movement and direction, and Grid Cells divide the spatial environment into a honeycomb hexagonal grid similar to the coordinate system on a map.
May-Britt Moser and Edvard Moser won the 2014 Nobel Prize in Physiology or Medicine for “discoveries of cells that constitute a positioning system in the brain,” ie the Grid Cell. However, some scientists speculate that beyond acting as the positioning system in the brain, Grid Cells also participate in vector calculations to assist route planning. This is the hypothesis that DeepMind wants to test with the help of AI techniques.
In the study, researchers used real world data and simulated movement trajectories of a large number of herbivorous rodents, then built models to learn these movements. The primary technique used was recurrent neural networks (RNN) with long short-term memory (LSTM), which memorized agents’ previous location, direction, and speed, and then integrated these with historical information to make the next move.
In short, DeepMind wants the program to learn to navigate like a rabbit. Surprisingly, the agent’s behaviour was similar to the neural activity patterns of the Grid Cell.
The team then applied deep reinforcement learning to examine if this grid-like structure could be used for vector-based navigation. The initial “grid network” was combined with a larger neural network architecture to form an agent, which was then trained with reinforcement learning in a virtual reality game environment. After model training the agent rose to a professional game player level in terms of navigation, taking shortcuts and discovering novel routes.
More importantly, when the generated grid units were silenced, the agent’s navigation capability became significantly less accurate in measurement of distance and direction.
Synced collected comments from scholars on Vector-based navigation using grid-like representations in artificial agents:
“The incredible discovery of grid cells showed that the brain creates maps of places by overlaying a spatial grid, something that would be very helpful in knowing where we are – providing us with something akin to a GPS signal.”
— Dharshan Kumaran in an interview with Wired, the author of this paper.
“It is interesting that the network, starting from very general computational assumptions that do not take into account specific biological mechanisms, found a solution to path integration that seems similar to the brain’s. That the network converged on such a solution is compelling evidence that there is something special about grid cells’ activity patterns that supports path integration.”
— Francesco Savelli and James Knierim, Neuroscientists at Johns Hopkins University.
“First, if the loss function of a neural network does not include regularization, it can not show the function of the grid cell. This discovery gives us a new way to approach the function of regularization. Second, the paper points out that the black-box characteristics of the deep neural network hinders the further analysis of the function of grid cell activity on path integration. This once again confirms the need to study the interpretability of neural networks at present.”
— Chunpeng Wu, a PhD student at Duke University.
“The study of position cells and grid cells is enlightening to artificial intelligence, especially the robot system. The position cell is actually the spatial index database, which describes the topological space; and the grid cell is the geometric calculator, which describes the Euclidean space. This organization is completely different from our current computer science technology and has a very strong advantage.”
— Fangde Liu, a former Research Associate at Imperial College and now Surgical AI CEO.
“One takeaway from this work (both the DeepMind paper and the Columbia paper, which was published first but will get many fewer citations) is that grid cells arise from path integration task. Path integration is great if you are nocturnal, but humans rely more strongly on vision.”
— Simon Kornblith, Google Brain Resident
Author: Mos Zhang| Editor: Tony Peng, Michael Sarazen
0 comments on “New DeepMind AI Learns to Navigate Like an Animal; Scientists React”