Google, Heidelberg University & NEC Propose Human Feedback for Real-World RL in NLP Systems
A new study proposes using human feedback and interaction logs to boost offline reinforcement learning (RL) in natural language processing (NLP).
Category: Nature Language Tech
Amazon Alexa AI’s ‘Language Model Is All You Need’ Explores NLU as QA
Amazon Alexa AI paper asks whether NLU problems could be mapped to question-answering (QA) problems using transfer learning.
Amazon’s BERT Optimal Subset: 7.9x Faster & 6.3x Smaller Than BERT
Amazon extracts an optimal subset of architectural parameters for BERT architecture by applying recent breakthroughs in algorithms for neural architecture search.
Google ‘mT5’ Pretrained Text-to-Text Transformer Achieves SOTA Performance on Multilingual Benchmarks
Google recently introduced mT5, a multilingual variant of its “Text-to-Text Transfer Transformer” (T5), pretrained on a new Common Crawl-based dataset covering 101 languages.
Facebook AI Model Directly Translates 100 Languages Without Using English Data
Facebook AI open-sourced a multilingual machine translation (MMT) model that translates between any pair of 100 languages without relying on English data.
Google, Stanford, & MIT Top NeurIPS 2020 Accepted Papers List
NeurIPS 2020 released its list of accepted papers this week with Google, Stanford, and MIT as the top affiliations.
Google AI ‘Looking to Listen’ Tech Enables Speech Enhancement on YouTube Stories in Seconds
Google AI has announced a new audiovisual speech enhancement feature in YouTube Stories (iOS) that enables creators to make better selfie videos by automatically enhancing their voices and reducing noise.
Google, Cambridge, DeepMind & Alan Turing Institute’s ‘Performer’ Transformer Slashes Compute Costs
A team from Google, University of Cambridge, DeepMind, and Alan Turing Institute have proposed a new type of Transformer dubbed Performer, based on a Fast Attention Via positive Orthogonal Random features (FAVOR+) backbone mechanism.
Google Approaches BERT-Level Performance Using 300x Fewer Parameters with Extension of Its New NLP model PRADO
The trimmed-down pQRNN extension to Google AI’s projection attention neural network PRADO compares to BERT on text classification tasks for on-device use.
Microsoft Gets Exclusive License for OpenAI’s GPT-3 Language Model
Microsoft announced today that it has teamed up with OpenAI to exclusively license the AI research institute’s GPT-3 language model.
Facebook AI Enables Automatic Closed Captions for Facebook Live
Facebook AI researchers and engineers just made live video content more accessible by enabling automatic closed captions for Facebook Live and Workplace Live.
Human Feedback Improves OpenAI Model Summarizations
OpenAI sets out to advance methods for training large-scale language models on objectives that more closely capture human preferences.
OpenAI ‘GPT-f’ Delivers SOTA Performance in Automated Mathematical Theorem Proving
OpenAI researchers introduce GPT-f, an automated prover and proof assistant for the Metamath formalization language.
New Multitask Benchmark Suggests Even the Best Language Models Don’t Have a Clue What They’re Doing
Researchers introduce a test covering topics such as elementary mathematics, designed to measure language models’ multitask accuracy.
Is OpenAI’s GPT-3 API Beta Pricing Too Rich for Researchers?
Although OpenAI hasn’t yet officially announced the GPT-3 pricing scheme, Branwen’s sneak peek has piqued the interest of the NLP community.
AMBERT: BERT with Multi-Grained Tokenization Achieves SOTA Results on English and Chinese NLU Tasks
AMBERT (A Multigrained BERT) leverages both fine-grained and coarse-grained tokenizations to achieve SOTA performance on English and Chinese language tasks.
Introducing Qian Yan, Baidu’s New Plan to Build 100 Chinese NLP Datasets in Three Years
Qian Yan, a plan to jointly develop the world’s largest Chinese natural language processing database.
Sepp Hochreiter on Parallels Between Attention Mechanisms and Modern Hopfield Networks
Researchers argue that “attention mechanism is the update rule of a modern Hopfield network with continuous states.”
Google Introduces NLP Model Understanding Tool
Google Research released a paper tackling this issue with a new open-source analytic platform: the Language Interpretability Tool (LIT).
How Smart is BERT? Evaluating the Language Model’s Commonsense Knowledge
Researchers dive deep into the large language model to discover how it encodes the structured commonsense knowledge it leverages on downstream commonsense tasks.
As Its GPT-3 Model Wows the World, OpenAI CEO Suggests ‘the Hype Is Way Too Much’
OpenAI’s 175 billion parameter language model GPT-3 has gone viral once again.
Google ‘BigBird’ Achieves SOTA Performance on Long-Context NLP Tasks
BigBird is shown to dramatically improve performance across long-context NLP tasks, producing SOTA results in question answering and summarization.
Applying Linearly Scalable Transformers to Model Longer Protein Sequences
Researchers proposed a new transformer architecture called “Performer” — based on what they call fast attention via orthogonal random features (FAVOR).
AI-Powered ‘Genderify’ Platform Shut Down After Bias-Based Backlash
Genderify is an AI-powered tool designed to identify a person’s gender by analyzing their name, username or email address.
Meet ByteDance AI’s Xiaomingbot: World’s First Multilingual and Multimodal AI News Agent
Researchers from ByteDance AILab and Shanghai Jiao Tong University have introduced Xiaomingbot, a multilingual and multimodal news reporter.
Grounded Language Learning: A Look at the Paper ‘Understanding Early Word Learning in Situated Artificial Agents’
This paper carefully engineers artificial-language learning experiments to replicate sources of information infants learn from and under what conditions.
Popular Microsoft Chatbot XiaoIce Gains Independence as a New Company Led by Di Li and Harry Shum
Microsoft announced it would spin off its chatbot business XiaoIce, with all associated technologies licensed to a newly formed independent company.
ProtTrans Delivers SOTA Pretrained Language Models for Proteins
Researchers launched the ProtTrans Project, which provides an outstanding model for protein pretraining.
PLATO-2: The State-Of-The-Art Open-Domain Chatbot in Chinese and English
PLATO-2, a open-domain chatbot model, can talk about anything in Chinese and English and engage in deep conversations.
Facebook & CMU Introduce TaBERT for Understanding Tabular Data Queries
TaBERT-powered neural semantic parsers showed performance improvements on the challenging benchmark WikiTableQuestions and demonstrated competitive performance on the text-to-SQL dataset Spider.
ACL 2020 Announces Best Paper & Test-Of-Time Awards
Organizers of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) today announced their Best Paper Awards.
Answering Multi-Hop Open-Domain Questions With Graph-Based Recurrent Retrieval
Researchers from the University of Washington, Salesforce Research and Allen Institute for Artificial Intelligence have introduced a graph-based method that retrieves reasoning paths to boost multi-hop open-domain question answering.
From Texts to Kitties: OpenAI’s GPT Language Model Tackles Image Generation
Large transformer-based language models trained on pixel sequences can generate coherent images without the use of labels.
A Look at Recent Advances in Google Translate
In a recent Google AI team blog post, researchers report on recent efforts and progress in the field of language translation, especially with resource-poor languages.
DeepMind Introduces ‘EATS’ – An Adversarial, End-to-End Approach to TTS
DeepMind researchers have developed EATS, a generative model trained adversarially in an end-to-end manner that achieves performance comparable to SOTA models.
DeepMind Says Syntactic Biases ‘Helped BERT Do Better’
Researchers add syntactic biases to determine whether and where they can help BERT achieve better understanding.
Bengio-Backed Startup Korbit Introduces STEM Intelligent Tutoring System
Canadian education technology startup Korbit Technologies has introduced a personalized AI-powered learning experience that it says can help all students learn faster and better in a cost-effective way.
Google Introduces BLEURT – a BERT-Based NLG Evaluation Metric
Google Research team proposes the automatic metric BLEURT which is based on the highly successful Google language model BERT.
ACL 2020 Announces Its 779 Accepted Papers
Organizers of the 58th annual meeting of the Association for Computational Linguistics (ACL) on Sunday announced the list of accepted papers for the world-leading natural language processing (NLP) conference.
Facebook’s Highly Efficient New Real-Time Text-To-Speech System Runs on CPUs
To deliver human-level voices to its platform’s billions of users while maintaining strict compute efficiency, Facebook AI researchers have deployed a new neural TTS system that works on CPU servers.