Jürgen Schmidhuber & Swiss AI Lab Team Boost Linear Transformers With Recurrent Fast Weight Programmers
A research team from the Swiss AI Lab IDSIA leverages fast weight programmers (FWPs) to advance linear transformers and explores the connection between linearised transformers and outer product-based FWPs to release the power of improved FWPs.