Neuromorphic computing is built on a brain-inspired silicon chip. It is widely regarded as a computing platform of the future — uniquely equipped to keep pace with machine learning’s increasingly demanding algorithms and exponentially growing scale of datasets.
Nobody knows neuromorphic computing better than Dr. Yiran Chen, who last week published a definitive paper on the subject in the very-large-scale-integration (VLSI) Journal Integration. Dr. Chen spent an entire year on Neuromorphic Computing’s Yesterday, Today, and Tomorrow, a paper that examines both the history and the future prospects of neuromorphic computing.
The paper details the relationship between neuromorphic computing and traditional artificial neural networks. It also addresses the dramatic rise of deep learning algorithms with questions such as “Why did deep learning models attract attention in 2006?” and “How did deep learning models affect neuromorphic computing?”
An Associate Professor at Duke University and Director of the Duke Center of Evolutionary Lab, Dr. Chen is mainly engaged in the collaborative design of AI-related hardware (nano-scale VLSI) and software (computing platforms). He is an IEEE Fellow for his contributions to memory technologies, and has published a book and more than 270 journal and conference papers.
Integration’s editor-in-chief Dr. Sheldon Tan tells Synced this paper will be both a very good reference for neuromorphic computing researchers, and an introduction for students who want to learn more about this important research field. The paper includes over 160 references.
“I am sure Dr. Chen’s paper will have many citations down the road,” says Dr. Tan.
The hottest direction in semiconductor hardware
The topic of neuromorphic computing seems to come up every few years in the scientific community, reflecting the need to constantly improve hardware performance to support advanced algorithm development.
The scale of artificial neural networks — especially deep learning models — is growing annually by an order of magnitude. The more advanced these algorithms and architectures become, the more data they can process. But hardware design struggles to keep up, which raises the question of how to increase a chip’s computing power while keeping it compact. Moore’s law says the computing power of a semiconductor can only double once per 18 months. Other hardware improvements must rely on new architectures and optimization.
Neuromorphic chips, which attempt to model the incredibly large and complex parallel processing power of the human brain, have the potential to overcome this hurdle due to their use of VLSI systems, which contain electronic analog circuits which mimic the nervous system in the human brain. If the individual components can change how they connect with each other in response to stimuli, then the computer chip can actually learn from its experiences just as an actual neural network does.
Neuromorphic computing can be roughly divided into three categories: analog, digital and mixed signal. Analog computing studies the biological properties of neural networks; digital computing focuses on accelerated chip design based on FPGA (field-programmable gate array) and ASIC (application-specific integrated circuit) technologies; and mixed signal combines the high accuracy and reliability of digital computing with the low power consumption of analog computing.
Advancing neuromorphic computing is challenging: new devices remain unstable for commercial use, there is no design that satisfies both generality and high performance, and the technology is constantly changing at a rapid rate.
Dr. Chen’s goal is to create stable models and simulations of emerging architectural designs to speed progress toward workable circuits.
Master of memory technologies
Dr. Chen was born in Henan, China, a beautiful inland province famous for Shaolin Temple and the home of Kung Fu. He received his Electronic Engineering Bachelor of Science with honors in 1998 and his Master of Science with honors in 2001 from Tsinghua University, and his PhD in Electrical and Computer Engineering from Purdue University in Indiana in 2005.
After graduation, Dr. Chen worked with world-leading software and IP company Synopsys as a senior research and development engineer, where he developed the award-winning statistical static timing analysis EDA tool “PrimeTimeVX”.
In 2007, Dr. Chen joined top US data storage company Seagate Technology, where he focused on emerging memory systems that rely on magnetism, electron spin, or nanoscale resistance. These magnetism-based memory systems perform better than electrons in storing information, offer better scalability, lower power consumption and faster performance.
Dr. Chen’s memory systems research achieved great success, spawning his three most-cited papers and a book co-authored with Dr. Hai Li at Duke University: “Nonvolatile Memory Design: Magnetic, Resistive, and Phase Changing”, published in 2011 by CRC Press.
At Seagate Advanced Technology Group Dr. Chen was a pioneer of “memristor,” a type of resistive memory similar to a human synapse that permits a neuron (or nerve cell) to pass an electrical or chemical signal to another neuron. This inspired Dr. Chen’s R&D on neuromorphic computing and deep learning accelerator.
In 2009, the subprime mortgage crisis hit many companies, Seagate among them. Dr. Chen took up a position at the University of Pittsburgh in 2010. It was here that his passion for neuromorphic computing would be ignited.
Dr. Chen’s serendipitous entry into neuromorphic computing
In the summer of 2010, Dr. Chen organized a panel in a nanometer-structure workshop that discussed the influence of the memristor on the new architecture. The panel included big names such as Google’s Distinguished Hardware Engineer Norm Jouppi, Dr. Robinson E. Pino from Department of Energy’s (DOE) Office of Science, and Air Force Research Laboratory Principal Electronics Engineer Qing Wu. The panel proposed the traditional Von Neumann architecture was not working effectively, and to match growing computational demands a new architecture was needed.
A year later, a chance reunion with one of the nanometer panel attendees brought Dr. Chen’s work into sync with the growing world of deep learning. “I realized that nanometer devices could be useful in deep learning models or as acceleration devices for artificial neural networks,” says Dr. Chen.
Dr. Chen came to believe that neuromorphic computing could disrupt the existing designs of semiconductors, nanometer devices, circuits, system architectures, design methods, and the development of algorithms. After his wife joined the University of Pittsburgh in 2012, the couple formed the Evolutionary Intelligence Lab. This year the lab transferred to Duke University and was upgraded to the Duke Center for Evolutionary Intelligence.
Born to be a teacher
Teaching and managing the lab consumes the majority of Dr. Chen’s time. He also has to travel to conferences and meet with research sponsors and collaborators. Over a recent six-month period Chen racked up 112,128 frequent flyer miles — equivalent to circling the equator 4.5 times.
Chen somehow also finds time to write. His WeChat blog is a vehicle for his sense of humour and expertise in areas such as memory systems and deep learning, and garners much attention from the AI community.
Coming from a high-school teacher’s family, Dr. Chen is passionate about teaching and education. While at the University of Pittsburgh he founded the “Cornell Cup”, a national competition of embedded system design for college students; and created an education YouTube channel.
In an interview with the IEEE community, Dr. Chen encouraged students to study engineering. “Engineer is a high-paying job with lots of fun and creativity. Nothing is cooler than when you can explain to your children how their video game systems work and watch their eyes fill with admiration,” says Dr. Chen, who loves playing Wii U and Xbox One with his two sons.
Dr. Chen has many research interests, from hardware to systems to algorithms, applications and so on. Building a tiny and power-efficient silicon chip with an artificial intelligence comparable to the capability of a human brain is by far his most challenging project, but Dr. Chen believes it has huge potential.
Implementing neuromorphic computing will however involve massive hardware overhauls across the industry. And that takes time and money.
“Neuromorphic computing is ready for the market,” says Dr. Chen, “the only question is whether the market is ready for neuromorphic computing.”
Journalist: Tony Peng | Editor: Michael Sarazen
0 comments on “Neuromorphic Computing: Yiran Chen’s Brainy Architecture of the Future”