Tag: Recurrent Neural Network

AI Machine Learning & Data Science Research

Jürgen Schmidhuber & Swiss AI Lab Team Boost Linear Transformers With Recurrent Fast Weight Programmers

A research team from the Swiss AI Lab IDSIA leverages fast weight programmers (FWPs) to advance linear transformers and explores the connection between linearised transformers and outer product-based FWPs to release the power of improved FWPs.

AI Machine Learning & Data Science Research

Yoshua Bengio Team’s Recurrent Independent Mechanisms Endow RL Agents With Out-of-Distribution Adaptation and Generalization Abilities

A research team from the University of Montreal and Max Planck Institute for Intelligent Systems constructs a reinforcement learning agent whose knowledge and reward function can be reused across tasks, along with an attention mechanism that dynamically selects unchangeable knowledge pieces to enable out-of-distribution adaptation and generalization.