Human Feedback Improves OpenAI Model Summarizations
OpenAI sets out to advance methods for training large-scale language models on objectives that more closely capture human preferences.
Category: Nature Language Tech
OpenAI ‘GPT-f’ Delivers SOTA Performance in Automated Mathematical Theorem Proving
OpenAI researchers introduce GPT-f, an automated prover and proof assistant for the Metamath formalization language.
New Multitask Benchmark Suggests Even the Best Language Models Don’t Have a Clue What They’re Doing
Researchers introduce a test covering topics such as elementary mathematics, designed to measure language models’ multitask accuracy.
Is OpenAI’s GPT-3 API Beta Pricing Too Rich for Researchers?
Although OpenAI hasn’t yet officially announced the GPT-3 pricing scheme, Branwen’s sneak peek has piqued the interest of the NLP community.
AMBERT: BERT with Multi-Grained Tokenization Achieves SOTA Results on English and Chinese NLU Tasks
AMBERT (A Multigrained BERT) leverages both fine-grained and coarse-grained tokenizations to achieve SOTA performance on English and Chinese language tasks.
Introducing Qian Yan, Baidu’s New Plan to Build 100 Chinese NLP Datasets in Three Years
Qian Yan, a plan to jointly develop the world’s largest Chinese natural language processing database.
Sepp Hochreiter on Parallels Between Attention Mechanisms and Modern Hopfield Networks
Researchers argue that “attention mechanism is the update rule of a modern Hopfield network with continuous states.”
Google Introduces NLP Model Understanding Tool
Google Research released a paper tackling this issue with a new open-source analytic platform: the Language Interpretability Tool (LIT).
How Smart is BERT? Evaluating the Language Model’s Commonsense Knowledge
Researchers dive deep into the large language model to discover how it encodes the structured commonsense knowledge it leverages on downstream commonsense tasks.
As Its GPT-3 Model Wows the World, OpenAI CEO Suggests ‘the Hype Is Way Too Much’
OpenAI’s 175 billion parameter language model GPT-3 has gone viral once again.
Google ‘BigBird’ Achieves SOTA Performance on Long-Context NLP Tasks
BigBird is shown to dramatically improve performance across long-context NLP tasks, producing SOTA results in question answering and summarization.
Applying Linearly Scalable Transformers to Model Longer Protein Sequences
Researchers proposed a new transformer architecture called “Performer” — based on what they call fast attention via orthogonal random features (FAVOR).
AI-Powered ‘Genderify’ Platform Shut Down After Bias-Based Backlash
Genderify is an AI-powered tool designed to identify a person’s gender by analyzing their name, username or email address.
Meet ByteDance AI’s Xiaomingbot: World’s First Multilingual and Multimodal AI News Agent
Researchers from ByteDance AILab and Shanghai Jiao Tong University have introduced Xiaomingbot, a multilingual and multimodal news reporter.
Grounded Language Learning: A Look at the Paper ‘Understanding Early Word Learning in Situated Artificial Agents’
This paper carefully engineers artificial-language learning experiments to replicate sources of information infants learn from and under what conditions.
Popular Microsoft Chatbot XiaoIce Gains Independence as a New Company Led by Di Li and Harry Shum
Microsoft announced it would spin off its chatbot business XiaoIce, with all associated technologies licensed to a newly formed independent company.
ProtTrans Delivers SOTA Pretrained Language Models for Proteins
Researchers launched the ProtTrans Project, which provides an outstanding model for protein pretraining.
PLATO-2: The State-Of-The-Art Open-Domain Chatbot in Chinese and English
PLATO-2, a open-domain chatbot model, can talk about anything in Chinese and English and engage in deep conversations.
Facebook & CMU Introduce TaBERT for Understanding Tabular Data Queries
TaBERT-powered neural semantic parsers showed performance improvements on the challenging benchmark WikiTableQuestions and demonstrated competitive performance on the text-to-SQL dataset Spider.
ACL 2020 Announces Best Paper & Test-Of-Time Awards
Organizers of the 58th Annual Meeting of the Association for Computational Linguistics (ACL) today announced their Best Paper Awards.
Answering Multi-Hop Open-Domain Questions With Graph-Based Recurrent Retrieval
Researchers from the University of Washington, Salesforce Research and Allen Institute for Artificial Intelligence have introduced a graph-based method that retrieves reasoning paths to boost multi-hop open-domain question answering.
From Texts to Kitties: OpenAI’s GPT Language Model Tackles Image Generation
Large transformer-based language models trained on pixel sequences can generate coherent images without the use of labels.
A Look at Recent Advances in Google Translate
In a recent Google AI team blog post, researchers report on recent efforts and progress in the field of language translation, especially with resource-poor languages.
DeepMind Introduces ‘EATS’ – An Adversarial, End-to-End Approach to TTS
DeepMind researchers have developed EATS, a generative model trained adversarially in an end-to-end manner that achieves performance comparable to SOTA models.
DeepMind Says Syntactic Biases ‘Helped BERT Do Better’
Researchers add syntactic biases to determine whether and where they can help BERT achieve better understanding.
Bengio-Backed Startup Korbit Introduces STEM Intelligent Tutoring System
Canadian education technology startup Korbit Technologies has introduced a personalized AI-powered learning experience that it says can help all students learn faster and better in a cost-effective way.
Google Introduces BLEURT – a BERT-Based NLG Evaluation Metric
Google Research team proposes the automatic metric BLEURT which is based on the highly successful Google language model BERT.
ACL 2020 Announces Its 779 Accepted Papers
Organizers of the 58th annual meeting of the Association for Computational Linguistics (ACL) on Sunday announced the list of accepted papers for the world-leading natural language processing (NLP) conference.
Facebook’s Highly Efficient New Real-Time Text-To-Speech System Runs on CPUs
To deliver human-level voices to its platform’s billions of users while maintaining strict compute efficiency, Facebook AI researchers have deployed a new neural TTS system that works on CPU servers.
AI2 & UW Researchers Propose ‘TL;DR’ Paper Summarization Task
A team from the Allen Institute for Artificial Intelligence and the University of Washington this week introduced TLDR generation, a new automatic summarization task for scientific papers.
Facebook Says Its ‘Blender’ Chatbot Is the Most Humanlike Ever
This is the first chatbot to blend a diverse set of conversational skills — including empathy, knowledge, and personality — together in one system.
AI21 Labs Asks: How Much Does It Cost to Train NLP Models?
How much is this going to cost? And what are the main factors affecting that price tag?
New Study Tracks NLP Development and Direction Through a ‘World Scope’
Researchers “posit that the universes of knowledge and experience available to NLP models can be defined by successively larger world scopes: from a single corpus to a fully embodied and social context.”
CMU, DeepMind & Google’s XTREME Benchmarks Multilingual Model Generalization Across 40 Languages
XTREME, a multi-task benchmark that evaluates cross-lingual generalization capabilities of multilingual representations across 40 languages and nine tasks.
Covid-Sanity Interface Elegantly and Efficiently Organizes Web COVID-19 Papers
Covid-Sanity, a web interface designed to navigate the flood of bioRxiv and medRxiv COVID-19 papers and make the research within more searchable and sortable.
BERTLang Helps Researchers Choose Between BERT Models
Researchers from Bocconi University have prepared an online overview of the commonalities and differences between language-specific BERT models and mBERT.
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Researchers propose a novel model compression approach to effectively compress BERT by progressive module replacing.
Retrieving Reasoning Paths for Answering Complex Questions
In an attempt to equip the TF-IDF-based retriever with a state-of-the-art neural reading comprehension model, researchers introduced a new graph-based recurrent retrieval approach.
Self-Adversarial Learning with Comparative Discrimination for Text Generation
Researchers have proposed a novel self-adversarial learning (SAL) paradigm for improving GANs’ performance in text generation.
Up Close and Personal With BERT – Google’s Epoch-Making Language Model
A recent Google Brain paper looks into Google’s hugely successful transformer network — BERT — and how it represents linguistic information internally.