Google AI Blog Introduces Lyra: A New Very Low-Bitrate Codec for Speech Compression
Google AI created Lyra, a high-quality, very low-bitrate speech codec that makes voice communication available even on the slowest networks.
Category: Research
Technical review of the newest machine intelligence research.
Better Than Capsules? Geoffrey Hinton’s GLOM Idea Represents Part-Whole Hierarchies in Neural Networks
A research team lead by Geoffrey Hinton has created an imaginary vision system called GLOM that enables neural networks with fixed architecture to parse an image into a part-whole hierarchy with different structures for each image.
Facebook AI’s Multitask & Multimodal Unified Transformer: A Step Toward General-Purpose Intelligent Agents
A research team from Facebook AI has proposed a Unified Transformer (UniT) encoder-decoder model that jointly trains on multiple tasks across different modalities and achieves strong performance on seven tasks with a unified set of model parameters.
Microsoft & Université de Montréal Researchers Leverage Measure Theory to Reveal the Mathematical Properties of Attention
A team from Microsoft and Université de Montréal proposes a new mathematical framework that uses measure theory and integral operators to achieve the goal of quantifying the regularity of the attention operation.
New Contextual Calibration Method Boosts GPT-3 Accuracy Up to 30%
A research team from UC Berkeley, University of Maryland and UC Irvine identifies pitfalls that cause instability in the GPT-3 language model and proposes a contextual calibration procedure that improves accuracy by up to 30 percent.
Panda Hub Chengdu Appointed National AI Innovation Zone Amongst Eight Other Cities
Ministry of Industry and Information technology appointed five more cities to the country’s National Artificial Intelligence Innovation Application Pilot Zones.
Visualize Your Model Errors! Microsoft Toolkit Identifies and Diagnoses ML Failures
A Microsoft team has introduced Error Analysis, a responsible AI toolkit designed to identify and diagnose errors in machine learning models.
BENDR for BCI: UToronto’s BERT-Inspired DNN Training Approach Learns From Unlabelled EEG Data
University of Toronto researchers propose a BERT-inspired training approach as a self-supervised pretraining step to enable deep neural networks to leverage newly and publicly available massive EEG (electroencephalography) datasets for downstream brain-computer-interface (BCI) applications.
Google & JHU Paper Explores and Categorizes Neural Scaling Laws
A research team from Google and Johns Hopkins University identifies variance-limited and resolution-limited scaling behaviours for dataset and model size in four scaling regimes.
Yann LeCun Hails MSA Transformer’s ‘Huge Progress’ in Protein Contact Prediction
UC Berkeley, Facebook AI Research and New York University researchers’ Multiple Sequence Alignments (MSA) Transformer surpasses current state-of-the-art unsupervised structure learning methods by a wide margin.
UC Berkeley & Google’s BoTNet Applies Self-Attention to CV Bottlenecks
Researchers from UC Berkeley and Google Research have introduced BoTNet, a “conceptually simple yet powerful” backbone architecture that boosts performance on computer vision (CV) tasks such as image classification, object detection and instance segmentation.
DeepMind Achieves High-Performance Large-Scale Image Recognition Without Batch Normalization
DeepMind has designed a family of Normalizer-Free ResNets (NFNets) that can be trained in larger batch sizes and stronger data augmentations and have set new SOTA validation accuracies on ImageNet.
Stanford University Deep Evolutionary RL Framework Demonstrates Embodied Intelligence via Learning and Evolution
Stanford researchers’ DERL (Deep Evolutionary Reinforcement Learning) is a novel computational framework that enables AI agents to evolve morphologies and learn challenging locomotion and manipulation tasks in complex environments using only low level egocentric sensory information.
DeepMind & UCL’s Alchemy Is a ‘Best-of-Both-Worlds’ 3D Video Game for Meta-RL
A research team from DeepMind and University College London have released Alchemy, a novel open-source benchmark for meta-RL research.
Transformers Scale to Long Sequences With Linear Complexity Via Nyström-Based Self-Attention Approximation
Researchers from the University of Wisconsin-Madison, UC Berkeley, Google Brain and American Family Insurance propose Nyströmformer, an adaption of the Nystrom method that approximates standard self-attention with O(n) complexity.
ICMI 2020 Best Paper | Gesticulator: A framework for semantically-aware speech-driven gesture generation
The research develop machine learning methods that enable virtual agents (such as avatars from a computer game) to communicate non-verbally.
Staying Abreast of Changing Language: DeepMind Explores Temporal Generalization in Language Models
In a bid to solve the temporal generalization problem of modern language models, a team of DeepMind researchers propose it’s time to develop adaptive language models that will remain up-to-date in our ever-changing world.
The Eyes Have It: 20 Million Images Make TEyeD World’s Largest Human Eye Dataset
A new study by University Tübingen introduces the world’s largest unified eye dataset with over 20 million human eye images captured using head-mounted eye trackers.
55 Researchers From 44 Institutions Propose GEM, a ‘Living Benchmark’ for NLG
To track progress in natural language generation (NLG) models, 55 researchers from more than 40 prestigious institutions have proposed GEM (Generation, Evaluation, and Metrics), a “living benchmark” NLG evaluation environment.
AAAI 2021 Best Papers Announced
The Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI-21) kicked off today as a virtual conference. The organizing committee announced the Best Paper Awards and Runners Up during this morning’s opening ceremony.
CMU Researchers Explore ‘Crazy Idea’ of Automating AI Paper Reviews
A bold Carnegie Mellon University (CMU) team recently explored the prospect of using AI to review AI papers.
New Transfer Learning Approach Summarizes Historical Texts in Modern Languages
Researchers from the University of Sheffield, Beihang University, and Open University’s Knowledge Media Institute have proposed a transfer learning approach that can automatically process historical texts at a semantic level to generate modern language summaries.
Georgia Tech & Facebook Tensor Train Approach Achieves 112x Size Reduction in DL Recommendation Models
A new study by the Georgia Institute of Technology and Facebook AI introduces TT-Rec, a way to drastically compress the size of memory-intensive Deep Learning Recommendation Models (DLRM) and make them easier to deploy at scale.
Google Brain Introduces Symbolic Programming + PyGlove Library to Reformulate AutoML
A recent study by the Google Brain Team proposes a new way of programming automated machine learning (AutoML) based on symbolic programming.
UmlsBERT: Clinical Domain Knowledge Augmentation of Contextual Embeddings Using the Unified Medical Language System Metathesaurus
UmlsBERT is a deep Transformer network architecture that incorporates clinical domain knowledge from a clinical Metathesaurus in order to build ‘semantically enriched’ contextual representations that will benefit from both the contextual learning and domain knowledge.
NVIDIA, UToronto, McGill & Vector Study Delivers Real-Time SDF Rendering & SOTA Complex Geometry Reconstruction
A new study by NVIDIA, University of Toronto, McGill University and the Vector Institute introduces an efficient neural representation that enables real-time rendering of high-fidelity neural SDFs for the first time while delivering SOTA geometry reconstruction quality.
ZeRO-Offload: Training Multi-Billion Parameter Models on a Single GPU
Researchers from University of California, Merced and Microsoft have introduced ZeRO-Offload, a novel heterogeneous DL training technology that enables training of multi-billion parameter models on a single GPU without any model refactoring.
Naver AI Lab Researchers Relabel 1.28 Million ImageNet Training Images
Researchers from Naver AI Lab says they’ve found a computationally efficient re-labelling strategy that fixes a significant flaw in popular image classification benchmark ImageNet.
AI Art Critic? New Dataset and Models Make Emotional Sense of Visual Artworks
Is it possible for machines to understand and integrate human emotions in visual arts? Meet ArtEmis, a new large-scale dataset of emotional reactions and explanations for visual artworks.
Tsinghua AI Report: China Applied for 389,571 AI Patents in the Past Ten Years
Analyzing papers and patent data collected from top AI journals and conferences, 44 in total from 2011 to 2020, Tsinghua University’s new AI development report revealed new statistical insights.
Drift with Devil: Security of Multi-Sensor Fusion based Localization in High-Level Autonomous Driving under GPS Spoofing
This paper is the first to systematically study the security of Multi-Sensor Fusion (MSF) based localization in high-autonomy Autonomous Vehicles (AVs). Researchers design FusionRipper, a novel and general attack that opportunistically captures and exploits take-over vulnerabilities.
CHI 2021 | ‘AI as Play’ Challenges the Productivity-Based Human-AI Interaction Paradigm
Researchers from Drexel University, Northeastern University and IT University Copenhagen explore how humans interact with AI in such contexts, with a focus on computer games.
Stanford Study Says Higher ImageNet Performance Does Not Improve Medical Image Interpretation
In a new paper, Stanford University researchers dispel the assumption that boosting deep learning model performance on ImageNet translates to better performance on medical imaging tasks.
Google Creates New SOTA Text-Image Generation Framework
Synced invited Dr. Linchao Zhu, a lecturer at the ReLER lab, University of Technology Sydney whose works focus on video representation learning, to share his thoughts on the paper Text-to-Image Generation Grounded by Fine-Grained User Attention.
McGill Researchers Use AI to Manage Extreme Weather Events
On January 18, in efforts to understand and manage extreme weather events, McGill University announced a new interdisciplinary project to combine deep learning (DL) with social network analysis (SNA).
ICLR 2021 | UT Austin Training-Free Framework Performs High-Quality NAS on ImageNet in Four GPU Hours
Researchers from the University of Texas, Austin have proposed a novel framework called Training-Free Neural Architecture Search (TE-NAS) for “training-free” neural architecture search.
Max Planck Institute & Facebook Model Performs Human Re-Rendering From a Single Image
In a new paper, researchers from Max Planck Institute for Informatics and Facebook Reality Labs propose an end-to-end trainable method that enables re-rendering of humans from one single image.
Will Humans Be Able to Control Superintelligent AI? New Study Says ‘No’
A paper in the Journal of Artificial Intelligence Research goes a step further, warning that it could become fundamentally impossible to control a superintelligent AI.
Google Brain’s Switch Transformer Language Model Packs 1.6-Trillion Parameters
Google Brain’s Switch Transformer language model packs a whopping 1.6 trillion parameters while effectively controlling computational cost. The model achieved a 4x pretraining speedup over a strongly tuned T5-XXL baseline.
ICLR 2021 Announces List of Accepted Papers
Of the 2997 submissions, 860 papers have made it to ICLR 2021, for an acceptance rate of 28.7 percent — slightly higher than last year’s 26.5 percent.