AI Technology & Industry Review
Probability theory is a mathematical framework for quantifying our uncertainty about the world, and is a fundamental building block in the study of machine learning. The purpose of this article is to provide the vocabulary and mathematics needed before applying probability theory to machine learning tasks.
A research team from Google Research propose a message-passing graph neural network that can explicitly model spatio-temporal relations, use either implicitly or explicitly representations of objects, and generalize previous structured models for video understanding.
A Google Research team accelerates Neural Radiance Fields’ rendering procedure for view-synthesis tasks, enabling it to work in real-time while retaining its ability to represent fine geometric details and convincing view-dependent effects.
A research team from Princeton University and Microsoft Research discover autonomous language-understanding agents are capable of achieving high scores even in the complete absence of language semantics, indicating that current RL agents for text-based games might not be sufficiently leveraging the semantic structure of game texts.
A research team from DeepMind and Alberta University proposes Policy-guided Heuristic Search (PHS), a novel search algorithm that uses both a heuristic function and a policy while offering guarantees on the search loss that relate to both the quality of the heuristic and the policy.
The Beijing Academy of Artificial Intelligence (BAAI) releases Wu Dao 1.0, China’s first large-scale pretraining model.
A research team from DeepMind proposes manipulation-independent representations (MIR) that can support successful imitation of behaviours demonstrated by previously unseen manipulator morphologies using only visual observations.
Researchers from Google, Georgia Tech and Emory University examine the semantic changes in abolitionist newspapers and explore whether the papers exhibited leadership or were followers, providing an overall picture of their network of semantic influence.
A research team from Hugging Face shows that prompting is indeed beneficial for fine-tuning pretrained language models, and that this benefit can be quantified as worth hundreds of data points on average across classification tasks.
A Microsoft research team provides concrete evidence showing that existing NLP models cannot robustly solve even the simplest of Math word problems, suggesting the hope that they might capably handle one-unknown arithmetic MWPs is untenable.
A Facebook AI research team explores strategies for convolutional neural network scaling, aiming to provide a framework for analyzing scaling strategies under various computational constraints.
Yann LeCun and a team of researchers propose Barlow Twins, a method that learns self-supervised representations through a joint embedding of distorted images, with an objective function that can make the embedding vectors almost identical while reducing redundancy between their components.
University of Oxford researchers propose COIN, a novel image compression method that stores the weights of an MLP overfitted to an image and outperforms JPEG at low bitrates even without entropy coding.
A research team from Facebook AI proposes a Contrastive Semi-supervised Learning (CSL) approach that synthesizes pseudo-labelling and contrastive losses to improve the stability of learned speech representations for automatic speech recognition (ASR).
A team from Qualcomm AI proposes the direct mapping of a deep neural network onto an optical quantum computer through the language of quantum field theory, paving the way for the future development of novel quantum neural network architectures.
A research team from Google and EPFL proposes a novel approach that sheds light on the operation and inductive biases of self-attention networks, and finds that pure attention decays in rank doubly exponentially with respect to depth.
A team from Huawei, ISCAS and UCAS propose the novel Transformer-iN-Transformer (TNT) for modelling both patch-level and pixel-level representations.
A team from Mila, University of Montreal, DeepMind, Waverly and Google Brain proposes Neural Production Systems, which serve to factorize entity-specific and rule-based information in rich visual environments.
A team from Facebook and Google has proposed LazyTensor — a technique for targeting domain-specific compilers without sacrificing define-by-run ergonomics.
A team from Google Research explores why most transformer modifications have not transferred across implementation and applications, and surprisingly discovers that most modifications do not meaningfully improve performance.
A novel module that effectively and efficiently propagates information through an arbitrarily long video, with constant complexity w.r.t. number of frames and linear complexity w.r.t. resolution.
A team from Max-Planck Institute for Intelligent Systems, ETH Zurich, Google Research Amsterdam, Mila and the University of Montreal make an effort to bring together causality and machine learning research programs, delineate implications of causality for machine learning and propose critical areas for future research.
A research team lead by Geoffrey Hinton has created an imaginary vision system called GLOM that enables neural networks with fixed architecture to parse an image into a part-whole hierarchy with different structures for each image.
A research team from Facebook AI has proposed a Unified Transformer (UniT) encoder-decoder model that jointly trains on multiple tasks across different modalities and achieves strong performance on seven tasks with a unified set of model parameters.
A team from Microsoft and Université de Montréal proposes a new mathematical framework that uses measure theory and integral operators to achieve the goal of quantifying the regularity of the attention operation.
A research team from UC Berkeley, University of Maryland and UC Irvine identifies pitfalls that cause instability in the GPT-3 language model and proposes a contextual calibration procedure that improves accuracy by up to 30 percent.
On February 3, Japanese automotive manufacturer Toyota has began to construct a high-tech city at the base of Mount Fuji.
South Korea’s Customs Service has partnered with the government-funded Institute for Basic Science (IBS) to conduct joint research on AI technology using customs trade data.
Dell Technologies has opened a new innovation facility in Singapore, to focus on R&D for edge computing, data analytics and augmented reality.
On February 18, Baidu released the 2020 full year and Q4 financial report, presenting revenue of RMB 107.1 Billion, and its net profit of RMB 22 billion.
On February 19, Beijing-based AI startup Unisound has decided halted its IPO application to the Shanghai Stalk Exchange.
A Microsoft team has introduced Error Analysis, a responsible AI toolkit designed to identify and diagnose errors in machine learning models.
University of Toronto researchers propose a BERT-inspired training approach as a self-supervised pretraining step to enable deep neural networks to leverage newly and publicly available massive EEG (electroencephalography) datasets for downstream brain-computer-interface (BCI) applications.
Apple has laid out the design characteristics of a new generic system that enables federated evaluation and tuning (FE&T) systems on end-user devices.
A research team from Google and Johns Hopkins University identifies variance-limited and resolution-limited scaling behaviours for dataset and model size in four scaling regimes.
On February 16, Goldman Sachs announced that it is launching an automated wealth management platform to invest client funds in a portfolio of stocks and bonds.
According to statement on February 12, Chinese AI chip unicorn Horizon Robotics announces a third time series C financing.
As part of the long-term strategy to “expand AI capabilities”, Swiss flavours and fragrance giant Givaudan announced that it will acquire the French company Myrissi.
UC Berkeley, Facebook AI Research and New York University researchers’ Multiple Sequence Alignments (MSA) Transformer surpasses current state-of-the-art unsupervised structure learning methods by a wide margin.
DeepMind has designed a family of Normalizer-Free ResNets (NFNets) that can be trained in larger batch sizes and stronger data augmentations and have set new SOTA validation accuracies on ImageNet.