AI Technology & Industry Review
A research team from Technical University of Munich, Google, Nvidia and LMU München proposes CodeTrans, an encoder-decoder transformer model which achieves state-of-the-art performance on six tasks in the software engineering domain, including Code Documentation Generation, Source Code Summarization, Code Comment Generation, etc.
A research and development team from ContinualAI, including a large group of researchers from KU Leuven, ByteDance AI Lab, University of California, New York University and other institutions, proposes Avalanche, an End-to-End Library for Continual Learning based on PyTorch.
A research team from University of Washington, Microsoft, DeepMind and Allen Institute for AI develop a method to convert pretrained transformers into efficient RNNs. The Transformer-to-RNN (T2R) approach speeds up generation and reduces memory cost.
A team from University of Michigan, MIT-IBM Watson AI Lab and ShanghaiTech University publishes two papers on individual fairness for ML models, introducing a scale-free and interpretable statistically principled approach for assessing individual fairness and a method for enforcing individual fairness in gradient boosting suitable for non-smooth ML models.
A research team from Google Research propose a message-passing graph neural network that can explicitly model spatio-temporal relations, use either implicitly or explicitly representations of objects, and generalize previous structured models for video understanding.
University of Oxford researchers propose COIN, a novel image compression method that stores the weights of an MLP overfitted to an image and outperforms JPEG at low bitrates even without entropy coding.
A research team from Facebook AI proposes a Contrastive Semi-supervised Learning (CSL) approach that synthesizes pseudo-labelling and contrastive losses to improve the stability of learned speech representations for automatic speech recognition (ASR).
A research team from Google and EPFL proposes a novel approach that sheds light on the operation and inductive biases of self-attention networks, and finds that pure attention decays in rank doubly exponentially with respect to depth.
University of Toronto researchers propose a BERT-inspired training approach as a self-supervised pretraining step to enable deep neural networks to leverage newly and publicly available massive EEG (electroencephalography) datasets for downstream brain-computer-interface (BCI) applications.
Apple has laid out the design characteristics of a new generic system that enables federated evaluation and tuning (FE&T) systems on end-user devices.
Researchers from the University of Sheffield, Beihang University, and Open University’s Knowledge Media Institute have proposed a transfer learning approach that can automatically process historical texts at a semantic level to generate modern language summaries.