BERTLang Helps Researchers Choose Between BERT Models
Researchers from Bocconi University have prepared an online overview of the commonalities and differences between language-specific BERT models and mBERT.
Category: Machine Learning & Data Science
Beyond Lipreading: Visual Speech Recognition Looks You in the Eye
A new study suggests that VSR models could perform even better if they used additional available visual information.
Chinese Researchers Use CNNs to Classify 3000-Year-Old Oracle Bone Scripts
The earliest evidence of China’s recorded history is found in the Shang dynasty (~1600 to 1046 BC), and this hasContinue Reading
Synthesized Paired Data Boosts Facial Manipulation
The model outperforms existing methods in image manipulation and offers researchers a possible solution to the scarcity of paired datasets.
Augmented Training Scheme Fixes CNN Texture Bias
Researchers proposed a new training scheme that targets this bias by controlling and exposing textural information slowly through the training process.
‘Train Large, Then Compress’ – UC Berkeley BAIR Improves Large Transformer Model Training and Inference
Researchers from the Berkeley Artificial Intelligence Research (BAIR) Lab at UC Berkeley explored the effect of Transformer model size on training and inference efficiency.
AutoCompress: SOTA Automatic DNN Pruning for Ultra-High Compression Rates
Researchers proposed an automatic structured pruning framework, AutoCompress, which adopts the 2018 ADMM-based weight pruning algorithm and outperforms previous automatic model compression methods while maintaining high accuracy.
AI-Based Drug Discovery Pipeline Generates Novel Drug Compounds against Coronavirus
A new study leverages an established AI-based drug discovery pipeline to produce molecular structures as part of the widening fight against the 2019-nCoV outbreak.
Yoshua Bengio and Team Introduce GNN Benchmarking Framework
Researchers propose a flexible GNN benchmarking framework that can also accommodate the needs of researchers to add new datasets and models.
Spotting Generated Images Just Got Much Easier
UC Berkeley and Adobe Research have introduced a “universal” detector that can distinguish real images from generated images regardless of what architectures and/or datasets were used for training.
CookGAN Generates Realistic Meal Images From an Ingredients List
Proposed by researchers from the Rutgers University and Samsung AI Center in the UK, CookGAN uses an attention-based ingredients-image association model to condition a generative neural network tasked with synthesizing meal images.
KaoKore: New Facial Dataset from Japanese Scrolls for ML
The KaoKore dataset includes 5552 RGB image files drawn from the 2018 Collection of Facial Expressions dataset of cropped face images from Japanese artworks.
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Researchers propose a novel model compression approach to effectively compress BERT by progressive module replacing.
Robot Lovers Rejoice! Fei-Fei Li Stanford Team Crowd-Sources World’s Largest Robot Manipulation Dataset
The crowdsourcing produced 111.25 hours of video from 54 non-expert demonstrators to build “one of the largest, richest, and most diverse robot manipulation datasets ever collected using human creativity and dexterity.”
Leap in Second-Order Optimization: Shampoo Runtime Boosted 40%
A Google-led research team has introduced a new method for optimizing neural network parameters that is faster than all common first-order methods on complex problems.
AI Enables Doctors to Diagnose COVID-19 Infection in Seconds
Fast and accurate diagnosis is critical on the front line, and now an AI-powered diagnostic assessment system is helping Hubei medical teams do just that.
Retrieving Reasoning Paths for Answering Complex Questions
In an attempt to equip the TF-IDF-based retriever with a state-of-the-art neural reading comprehension model, researchers introduced a new graph-based recurrent retrieval approach.
Aerial Visual Search of the Continental USA in 0.1 Seconds
The proposed system is capable of searching the continental United States at 1 -meter pixel resolution, corresponding to approximately 2 billion images, in around 0.1 seconds.
MonoLayout | Bird’s-Eye Layout Estimation from A Single Image
MonoLayout, a practical deep neural architecture that takes just a single image of a road scene as input and outputs an amodal scene layout in bird’s-eye view.
Bengio and Mila Researchers Use GAN Images to Illustrate Impact of Climate Change
In a bid to raise awareness of the threats posed by climate change, the Mila team recently published a paper that uses GANs to generate images of how climate events may impact our environments — with a particular focus on floods.
Self-Adversarial Learning with Comparative Discrimination for Text Generation
Researchers have proposed a novel self-adversarial learning (SAL) paradigm for improving GANs’ performance in text generation.
A look at The Case for Bayesian Deep Learning
Bayesian inference meanwhile leverages Bayes’ theorem to update the probability of a hypothesis as additional data becomes available. How can Bayesian inference benefit deep learning models?
DeepMind Releases New JAX Libraries for Neural Networks and Reinforcement Learning
DeepMind announced yesterday the release of Haiku and RLax — new JAX libraries designed for neural networks and reinforcement learning respectively.
Introduction to Deep Learning for Graphs and Where It May Be Heading
Researchers from Italy’s University of Pisa present a clear and engaging tutorial on the main concepts and building blocks involved in neural architectures for graphs.
Anime-zing GANILLA for Ghibli Style Image Transfer
Researchers have proposed a novel generator network specialized on the illustrations in children’s books.
Geoffrey Hinton & Google Brain Unsupervised Learning Algorithm Improves SOTA Accuracy on ImageNet by 7%
Researchers have proposed a simple but powerful “SimCLR” framework for contrastive learning of visual representations.
Up Close and Personal With BERT – Google’s Epoch-Making Language Model
A recent Google Brain paper looks into Google’s hugely successful transformer network — BERT — and how it represents linguistic information internally.
Allen Institute’s New ‘Computer Vision Explorer’ Lets Researchers Demo SOTA CV Models
The tool enables researchers to try, compare, and evaluate models to decide which work best on their datasets or for their research purposes.
‘ClearGrasp’ Helps Robots See and Grasp Transparent Objects
Google teamed up with researchers from Synthesis AI and Columbia University to introduce a deep learning approach called ClearGrasp as a first step to teaching machines how to “see” transparent materials.
Google Brain & CMU Semi-Supervised ‘Noisy Student’ Achieves 88.4% Top-1 Accuracy on ImageNet
Researchers from Google Brain and Carnegie Mellon University have released models trained with a semi-supervised learning method called “Noisy Student” that achieve 88.4 percent top-1 accuracy on ImageNet.
Unsupervised Approach for GAN Interpretability Through Semantic Direction Discovery
Researchers have introduced the first unsupervised learning approach for identifying interpretable semantic directions in the latent space of generative adversarial network (GAN) models.
17 Billion Parameters! Microsoft DeepSpeed Breeds World’s Largest NLP Model
Deep learning models are getting larger and larger to meet the demand for better and better performance. Meanwhile, the timeContinue Reading
Beyond SPADE: Introducing Semantic Region-Adaptive Normalization for Image Synthesis
Researchers introduced semantic region-adaptive normalization (SEAN), a simple but effective building block for conditional Generative Adversarial Networks (cGAN).
AAAI 2020 | A Turning Point for Deep Learning? Hinton, LeCun, and Bengio Might Have Different Approaches
The Godfathers of AI and 2018 ACM Turing Award winners Geoffrey Hinton, Yann LeCun, and Yoshua Bengio shared a stage in New York on Sunday night at an event organized by the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020).
Battling the Coronavirus: Alibaba and Baidu AI Accelerate Vaccine and Drug R&D
The crucial step now is to develop matching vaccines and drugs to uproot its existence, and China’s big tech companies have stepped up to help.
Batchboost: Regularization for Stabilizing Training With Resistance to Underfitting & Overfitting
Batchboost is a simple technique to accelerate ML model training by adaptively feeding mini-batches with artificial samples which are created by mixing two examples from the previous step – in favor of pairing those that produce the difficult one.
Nature Bid to Boost Paper Review Transparency Applauded by ML Researchers
Leading scientific publication Nature announced it is launching a trial starting this week that will give authors of newly published papers the option of appending contents of the discussions they’ve had with and reports they’ve received from their reviewers.
Sing Along! Japanese Researchers Develop Multispeaker Corpus for Singing Voice Synthesis
In an effort to enrich resources for multispeaker singing-voice synthesis, a team of researchers from the University of Tokyo has developed a Japanese multispeaker singing-voice corpus.
Radioactive Data: Facebook AI Knows Where You Got Your Training Dataset
Researchers proposed a “radioactive data” technique for subtly marking images in a dataset to help researchers later determine whether they were used to train a particular model.
Machine Unlearning: Fighting for the Right to Be Forgotten
In a new paper, researchers from the University of Toronto, Vector Institute, and University of Wisconsin-Madison propose SISA training, a new framework that helps models “unlearn” information by reducing number of updates that need to be computed when data points are removed.