Synced Tradition and Machine Learning Series | Part 2: Optimization Basics
This is the second in a special Synced series of introductory articles on traditionally theoretical fields of studies and their impact on modern-day machine learning.
Tag: Neural Networks
For Its Latest Trick, OpenAI’s GPT-3 Generates Images From Text Captions
OpenAI has trained a neural network called DALL·E that creates images from text captions for a wide range of concepts expressible in natural language.
URI Researchers Trained CNN Networks to ID mosquitos
Rapid and accurate identification of mosquitoes that transmit human pathogens such as malaria is an essential part of mosquito-borne disease surveillance.
ICLR 2021 Submission: Deeper VAEs Excel on Natural Image Benchmarks
ICLR 2021 submitted paper proposes efficient VAEs that outperform PixelCNN-based autoregressive models in log-likelihood on natural image benchmarks.
A Look at IGLOO: Slicing the Features Space to Represent Sequences
The ReDNA Labs research team has devised a new lightweight approach called IGLOO which allows to deal with sequences up to 25,000 steps long.
Facebook One-Shot, On-Device Model Efficiently Transforms Smartphone Pics Into 3D Images
Facebook has introduced a model that turns common two-dimensional pictures into 3D photos.
DeepMind Explores Generalization and Efficiency in Algorithm Design
DeepMind introduced a new approach designed to improve the generalizability and efficiency of algorithms represented by neural networks.
Will Artificial Brain Synapses & Neuromorphic Computing Open the Next AI Hardware Frontier?
There are one trillion synapses in a cubic centimeter of the brain. If there is such a thing as General AI, it would probably require one trillion synapses.
ICLR 2020 | A Look at Three Interesting Papers on the Robustness of Neural Networks
A look at three ICLR papers to look at that fall under the topic of robustness
Neural Network AI is the Future of the Translation Industry
Recent advances in deep learning, also known as using neural machine learning, has proven to achieve state of the art results in machine translation.
Google ‘Data Echoing’ Accelerates Neural Network Training
A “data echoing” technique that enables these time-consuming upstream training stages to also benefit from accelerators.
House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation
This paper proposes a novel graph-constrained generative adversarial network, whose generator and discriminator are built upon relational architecture.
AI21 Labs Asks: How Much Does It Cost to Train NLP Models?
How much is this going to cost? And what are the main factors affecting that price tag?
Amazon Introduces ResNeSt: Strong, Split-Attention Networks
The latest ResNet improvement comes courtesy researchers from Amazon and UC Davis, who unveiled their Split-Attention Networks, ResNeSt.
Google & DeepMind Unify Normalization and Activation Layers, Discover ‘EvoNorms’
Researchers unify DNN normalization layers and activation functions into a single computation graph.
Augmented Training Scheme Fixes CNN Texture Bias
Researchers proposed a new training scheme that targets this bias by controlling and exposing textural information slowly through the training process.
DeepMind Releases New JAX Libraries for Neural Networks and Reinforcement Learning
DeepMind announced yesterday the release of Haiku and RLax — new JAX libraries designed for neural networks and reinforcement learning respectively.
Anime-zing GANILLA for Ghibli Style Image Transfer
Researchers have proposed a novel generator network specialized on the illustrations in children’s books.
AAAI 2020 Best Papers; Turing Award Winners See a Turning Point for Deep Learning; MIT Reveals Voting App Vulnerabilities
Synced Global AI Weekly February 16th
Facebook HiPlot ‘Makes Understanding High Dimensional Data Easy’
Facebook’s new HiPlot is a lightweight interactive visualization tool that takes this further, using parallel plots to discover correlations and patterns in such high-dimensional data.
DeepMind & UCL Introduce New Model and Test Set for Inference
Now, DeepMind and University College London (UCL) have introduced a new deep network called MEMO which matches SOTA results on Facebook’s bAbI dataset for testing text understanding and reasoning, and is the first and only architecture capable of solving long sequence novel reasoning tasks.
Google Introduces Flax: A Neural Network Library for JAX
Google recently introduce Flax - a neural network library for JAX that is designed for flexibility.
AI Listens to Panda Love Sounds, Predicts Mating Success
Researchers trained a neural network, CGANet (Convolution module, bidirectional Gated Recurrent Unit module, Attention module), to automate the mating success prediction process for pandas based on their vocal sounds.
Facebook AI | Text-Based Generation of Interactive Worlds
Researchers recently proposed a new machine learning method for worldbuilding based on content from LIGHT, a research environment open-sourced by Facebook comprising crowd-sourced game locations, characters, and objects, etc.
Gaussian Error Linear Unit Activates Neural Networks Beyond ReLU
Results of the various experiments show GELU consistently has the best performance compared with ReLU and ELU, and can be considered a viable alternative to previous nonlinear approaches.
King’s and University of Surrey | Deep Learning Improves Structural Health Monitoring of Civil Infrastructure
This research demonstrated that deep learning can contribute to the traditional discipline with much better performance than existing methods.
Big RNNs Achieve SOTA Performance in Video Prediction
It’s not as easy as one might imagine to train an AI model to accurately predict what a human will do next, even when they are interacting with a relatively simple object like a ball.
Can Bots Surpass the ‘Realism’ of Human Dialogue?
Microsoft’s new tunable gigaword-scale neural network DialogGPT is a virtual master of conversation that outperforms strong baseline systems in generating relevant and context-consistent responses and attains near human level performance in conversational response generation tasks.
Google T5 Explores the Limits of Transfer Learning
Synced invited Samuel R. Bowman, an Assistant Professor at New York University who works on artificial neural network models for natural language understanding, to share his thoughts on the “Text-to-Text Transfer Transformer” (T5) framework.
ProtoPNet Recognizes Birds and Shows Us How in Real Time
Exciting new research from Duke University introduces ProtoPNet, a deep learning network that can explain how it distinguishes a pigeon from a partridge in real time.
AI Visualizes Your Post Cosmetic Surgery Appearance
Cosmetic surgery companies are turning to big data, facial recognition, neural networks, adversarial learning and deep learning technologies that can assess the human face and generate outcomes for a specific procedures to guide patients to their best surgical options.
Quantum Chemistry Breakthrough: DeepMind Uses Neural Networks to Tackle Schrödinger Equation
DeepMind researchers have brought quantum Monte Carlo (QMC) to a higher level with the Fermionic Neural Network — or Fermi Net — a neural network with more flexibility and higher accuracy.
OpenAI Method Evaluates Model Defense Against Unforeseen Adversarial Examples
To ramp up the robustness of neural networks, researchers from OpenAI have introduced a novel method that evaluates how well a neural network classifier performs against adversarial attacks that were not seen during their training.
Of Mice and Machines: Can AI Read Rodents’ Minds?
Researchers from Two Six Labs and Stanford Schnitzer Lab have developed a deep learning system designed to explore the workings of the mouse mind and predict behavior by processing brain-based electrical activity with a neural network.
Does Deep Learning Still Need Backpropagation?
Now, researchers from the Victoria University of Wellington School of Engineering and Computer Science have introduced the HSIC (Hilbert-Schemidt independence criterion) bottleneck as an alternative to backpropagation for finding good representations.
Facebook AI Memory Layer Boosts Network Capacity by a Billion Parameters
Recently, Facebook AI Research (FAIR) researchers introduced a structured memory layer which can be easily integrated into a neural network to greatly expand network capacity and the number of parameters without significantly changing calculation cost.
Are Weights Really Important to Neural Networks?
Architecture and weights are two essential considerations for artificial neural networks. Architecture is akin to the innate human brain, and contains the neural network’s initial settings such as hyperparameters, layers, node connections (or wiring), etc.
DeepMind AI Reaches Human-Level Performance in Quake III Arena
DeepMind is a trailblazer in the trending computer vs humans gaming research space. Following milestone victories against human pros on the board game Go and video game StarCraft II, the Google-owned research company has now pitted their new AI system against humans in the first-person shooter multiplayer video game Quake III Arena.
Geoffrey Hinton Leads Google Brain Representation Similarity Index Research Aiming to Understand Neural Networks
A Google Brain research team led by Turing Award recipient Geoffrey Hinton recently published a paper that presents an effective method for measuring the similarity of representations learned by deep neural networks.
Google I/O 2019 | Geoffrey Hinton Says Machines Can Do Anything Humans Can
Artificial intelligence is closing the gap on humans. Machines are rapidly honing their skills in object recognition and natural language interaction, and advanced AI agents have already beat human champions in board and video games and even debates.