AI Technology & Industry Review
Global machine intelligence updates.
A research team from McGill University, Mila – Quebec AI Institute and Facebook AI proposes novel metrics and perturbation functions to detect, quantify and compare trade-offs between robustness and faithfulness in NMT systems, both on the corpus level and with particular examples.
An IBM research team proposes four multilingual adversarial attack strategies and attacks seven languages in a zero-shot setting on large multilingual pretrained language models (e.g. MBERT), reducing average performance by up to 85.6 percent.
On April 21, the European Commission proposed its first comprehensive legal framework on AI, in effort to “address the risks of AI and position Europe to play a leading role globally”.
A research team from UC Berkeley and Carnegie Mellon University proposes a task-agnostic reinforcement learning method that reduces the task-specific engineering required for domain randomization of both visual and dynamics parameters.
On April 19, Google Cloud and Siemens announced new cooperation to optimize factory processes and improve productivity on the shop floor. Siemens intends to integrate Google Cloud’s leading data cloud and artificial intelligence/machine learning (AI/ML) technologies with its factory automation solutions.
According to Chinese media report on April 19, there have been at least 21 AI chips investments involving 17 companies since the beginning of 2021, totaling approx. RMB 20 billion (USD 3.1 billion).
A research team from Rice University, IBM and USC combine compressed sensing, non-convex optimization and acceleration techniques to introduce a new algorithm — Momentum Inspired Factored Gradient Descent (MiFGD) — that pushes QST beyond current capabilities.
A research team from DeepMind introduces Anakin and Sebulba, two architectures that demonstrate reinforcement learning platforms based on TPUs can efficiently deliver exceptional performance at scale and with low cost.
A research team from ETH Zurich leverages existing spike-based learning circuits to propose a biologically plausible architecture that is highly successful in classifying distinct and complex spatio-temporal spike patterns. The work contributes to the design of ultra-low-power mixed-signal neuromorphic processing systems capable of distinguishing spatio-temporal patterns in spiking activity.
On April 15, self-driving truck company TuSimple raised USD 1.35 billion with its NASDAQ IPO, with company valuation hitting USD 8.5 billion at high point.
A research team from NVIDIA, Stanford University and Microsoft Research propose a novel pipeline parallelism approach that improves throughput by more than 10 percent with a comparable memory footprint, showing such strategies can achieve high aggregate throughput while training models with up to a trillion parameters.
A research team from ETH and UC Berkeley proposes a Deep Reward Learning by Simulating the Past (Deep RLSP) algorithm that represents rewards directly as a linear combination of features learned through self-supervised representation learning and enables agents to simulate human actions backwards in time to infer what they must have done.
On April 12, Microsoft Corp announced the acquisition of Nuance for USD 19.7 billion, which is the largest acquisition after its LinkedIn deal for USD 26 billion in 2016.
On April 12, the Swiss National Supercomputing Centre (CSCS), Hewlett Packard Enterprise (HPE) and NVIDIA announced that they are creating what is expected to be the world’s most powerful AI-capable supercomputer.
A research team from Google Brain and New York University says the Natural Language Understanding (NLU) evaluation system is “broken” and proposes four criteria for improving NLU benchmarks.
A research team from IBM introduces two systems for predicting information type: The TypeSuggest module, an unsupervised system designed to generate types for a set of seed query terms input by the user; and an Answer Type prediction module for predicting the correct answer type for user-provided questions.
A research team from Technical University of Munich, Google, Nvidia and LMU München proposes CodeTrans, an encoder-decoder transformer model which achieves state-of-the-art performance on six tasks in the software engineering domain, including Code Documentation Generation, Source Code Summarization, Code Comment Generation, etc.
On April 8, Chinese robotics startup CloudMinds finished series B+ round of funding with RMB 1 Billion (approx. USD 150 million) investment led by SOEs Shanghai Urban Construction Investment and Shanghai Guosheng Group.
A research and development team from ContinualAI, including a large group of researchers from KU Leuven, ByteDance AI Lab, University of California, New York University and other institutions, proposes Avalanche, an End-to-End Library for Continual Learning based on PyTorch.
On April 7, media reports show that China’s top-tier electric vehicle manufacturer, XPeng or Xiaopeng Motors, has commenced R&D for owned self-driving chips.
A research team from University of Washington, Microsoft, DeepMind and Allen Institute for AI develop a method to convert pretrained transformers into efficient RNNs. The Transformer-to-RNN (T2R) approach speeds up generation and reduces memory cost.
A team from University of Michigan, MIT-IBM Watson AI Lab and ShanghaiTech University publishes two papers on individual fairness for ML models, introducing a scale-free and interpretable statistically principled approach for assessing individual fairness and a method for enforcing individual fairness in gradient boosting suitable for non-smooth ML models.
A Yann LeCun team proposes dictionary learning to provide detailed visualizations of transformer representations and insights into semantic structures such as word-level disambiguation, sentence-level pattern formation, and long-range dependency captured by transformers.
Probability theory is a mathematical framework for quantifying our uncertainty about the world, and is a fundamental building block in the study of machine learning. The purpose of this article is to provide the vocabulary and mathematics needed before applying probability theory to machine learning tasks.
A research team from Google Research propose a message-passing graph neural network that can explicitly model spatio-temporal relations, use either implicitly or explicitly representations of objects, and generalize previous structured models for video understanding.
A Google Research team accelerates Neural Radiance Fields’ rendering procedure for view-synthesis tasks, enabling it to work in real-time while retaining its ability to represent fine geometric details and convincing view-dependent effects.
A research team from Princeton University and Microsoft Research discover autonomous language-understanding agents are capable of achieving high scores even in the complete absence of language semantics, indicating that current RL agents for text-based games might not be sufficiently leveraging the semantic structure of game texts.
A research team from DeepMind and Alberta University proposes Policy-guided Heuristic Search (PHS), a novel search algorithm that uses both a heuristic function and a policy while offering guarantees on the search loss that relate to both the quality of the heuristic and the policy.
A research team from ETH Zurich, SIB Swiss Institute of Bioinformatics, and KU Leuven proposes Topological Graph Layer (TOGL), a new type of graph neural network layer capable of leveraging the multi-scale topological information of input graphs.
Tsinghua & MIT researchers break the stereotype that GPTs can generate but not understand language, showing that GPTs can compete with BERT models on natural language understanding tasks using a novel P-tuning method that can also improve BERT performance in both few-shot and supervised settings.
A research team from CMU, Oxford and Facebook AI proposes a vision-language model that, when trained on a source language, can be applied to different languages without additional annotated training data.
The Beijing Academy of Artificial Intelligence (BAAI) releases Wu Dao 1.0, China’s first large-scale pretraining model.
A research team from DeepMind proposes manipulation-independent representations (MIR) that can support successful imitation of behaviours demonstrated by previously unseen manipulator morphologies using only visual observations.
Researchers from Google, Georgia Tech and Emory University examine the semantic changes in abolitionist newspapers and explore whether the papers exhibited leadership or were followers, providing an overall picture of their network of semantic influence.
A research team from Hugging Face shows that prompting is indeed beneficial for fine-tuning pretrained language models, and that this benefit can be quantified as worth hundreds of data points on average across classification tasks.
A Microsoft research team provides concrete evidence showing that existing NLP models cannot robustly solve even the simplest of Math word problems, suggesting the hope that they might capably handle one-unknown arithmetic MWPs is untenable.
A Facebook AI research team explores strategies for convolutional neural network scaling, aiming to provide a framework for analyzing scaling strategies under various computational constraints.
Yann LeCun and a team of researchers propose Barlow Twins, a method that learns self-supervised representations through a joint embedding of distorted images, with an objective function that can make the embedding vectors almost identical while reducing redundancy between their components.
University of Oxford researchers propose COIN, a novel image compression method that stores the weights of an MLP overfitted to an image and outperforms JPEG at low bitrates even without entropy coding.
A research team from Facebook AI proposes a Contrastive Semi-supervised Learning (CSL) approach that synthesizes pseudo-labelling and contrastive losses to improve the stability of learned speech representations for automatic speech recognition (ASR).