ICLR 2018’s Best Papers: Variant Adam, Spherical CNNs, and Meta-Learning

Leading machine learning conference International Conference on Learning Representations (ICLR) has named its best research papers of the last year: On the convergence of Adam and Beyond, Spherical CNNs, and Continuous Adaptation via Meta-learning in Nonstationary and Competitive Environments.

Launched in 2013, the ICRL has grown to a world-class conference for machine learning researchers and engineers. ICRL 2018 received 935 papers — double last year’s total — and 337 papers were accepted.

In On the Convergence of Adam and Beyond, Google New York proposes the new variant Adam, a gradient descent optimization algorithm introduced in ICLR 2015.

Gradient Descent is one of the most popular algorithm types for optimizing neural networks, but struggles with a convergence issue in non-convex settings that makes optimizations ineffectual. The Google paper suggests a new Adam algorithm which it says fixes the problem and improves the empirical performance.

Researchers at the University of Amsterdam proposed Spherical Convolutional Neural Networks (CNNs) which can analyze spherical images, a technique in wide demand for drones, robots, autonomous cars, molecular regression problems, and global weather and climate modelling. The paper demonstrates that spherical CNNs can be efficiently applied to 3D model recognition and atomization energy regression.

In Continuous Adaptation via Meta-learning in Nonstationary and Competitive Environments, top academic institutes CMU, UMass Amherst, OpenAI, and UC Berkeley jointly developed a simple gradient-based meta-learning algorithm that can adapt to dynamically changing and adversarial environments.

Over the past few years, reinforcement learning (RL) has successfully enabled machines to outperform humans in tasks ranging from Atari video games to the ancient and complex Chinese board game Go. However, the AI technique is not adaptable to non-stationary environments, for example a multiplayer game with high randomness. Meta-learning, the so-called learning-to-learn method, can compensate for RL’s weakness.

This paper also introduced a new multi-agent competitive environment, RoboSumo, for more effective training of meta-learning algorithms.

ICLR 2018 runs April 30 to May 3 at the Vancouver Convention Centre in Vancouver, Canada.