Peeking Inside DNNs With Information Theory
Deep learning model performance has taken huge strides, allowing researchers to tackle tasks which were simply not possible for machines less than a decade ago.
Tag: machine translation
Google Researchers Add Attention to Augment Convolutional Neural Networks
A group of Google researchers led by Quoc Le — the AI expert behind Google Neural Machine Translation and AutoML — have published a paper proposing attention augmentation. In experiment results, the novel two-dimensional relative self-attention mechanismfor image classification delivers “consistent improvements in image classification.”
Facebook Boosts Cross-Lingual Language Model Pretraining Performance
Facebook researchers have introduced two new methods for pretraining cross-lingual language models (XLMs). The unsupervised method uses monolingual data, while the supervised version leverages parallel data with a new cross-lingual language model.
Free AI Study Materials to Start New Year Resolution
Synced Global AI Weekly January 6th
Tsinghua University Publishes Comprehensive Machine Translation Reading List
Tsinghua Natural Language Processing Group (THUNLP) has published a great reading list for any budding AI researchers whose New Year’s resolution is to study machine translation. The list compiles the most influential machine translation papers from the past 30 years, spotlighting the 10 most important contributions to the development of machine translation.
Talk Review: Deep Learning: Practice and Trends, NIPS 2017 – Part I
Deep Learning has become an essential toolbox which is used in a wide variety of applications, research labs, industries, etc. In this tutorial given at NIPS 2017, the speakers provide a set of guidelines which will help newcomers to the field understand the most recent and advanced models and their application to diverse data modalities.
Google’s AI-Powered Navlekha Boosts Indian Language Online Content
At the Google For India 2018 conference in New Delhi yesterday Google launched its AI platform Navlekha, which enables publishers to make offline Indian language content fully editable and streamlines the online publishing process.
Focus on AI at Facebook F8; PyTorch 1.0 Released (Updated)
Facebook generally uses its F8 Developer Conference to introduce new website features. This year however there was a distinct emphasis on “AI,” with the term mentioned more than ever in the event’s keynote speech.
Microsoft AI Achieves Milestone in Machine Translation
Microsoft researchers in the US and Asia sent a shockwave through the AI community today with their paper Achieving Human Parity on Automatic Chinese to English News Translation, which introduces a neural machine translation system they say equals the performance of human experts in Chinese-to-English translation.
Neural System Combination for Machine Translation
To combine the advantages of these two methods, the authors of this paper first adapts the multi-source NMT model, by employing different encoders to capture the semantics of the source language, then the decoder is used to generate the final output by the multiple context vector representations coming from the encoder.
A Brief Overview of Attention Mechanism
Attention is simply a vector, often the outputs of dense layer using softmax function.