Up Close and Personal With BERT – Google’s Epoch-Making Language Model
A recent Google Brain paper looks into Google’s hugely successful transformer network — BERT — and how it represents linguistic information internally.
Category: Nature Language Tech
17 Billion Parameters! Microsoft DeepSpeed Breeds World’s Largest NLP Model
Deep learning models are getting larger and larger to meet the demand for better and better performance. Meanwhile, the timeContinue Reading
DeepMind & UCL Introduce New Model and Test Set for Inference
Now, DeepMind and University College London (UCL) have introduced a new deep network called MEMO which matches SOTA results on Facebook’s bAbI dataset for testing text understanding and reasoning, and is the first and only architecture capable of solving long sequence novel reasoning tasks.
Meet Meena: A Truly Chatty Chatbot
One of a new breed of open-domain chatbots designed to engage in conversations across any topic, Meena’s free and natural conversational abilities are closing the gap on human performance.
Facebook AI mBART: The Tower of Babel’s Silicon Solution
Facebook AI researchers have further developed the BART model with the introduction of mBART.
AraNet: New Deep Learning Toolkit for Arabic Social Media
A team of researchers from the Natural Language Processing Lab at the University of British Columbia in Canada have proposed AraNet, a deep learning toolkit designed for Arabic social media processing.
Say Woof? AI in Animal Language Translation
No matter whether we’re sharing our lives with beloved household pets or protecting wildlife in a remote location, wouldn’t it be wonderful if we could somehow lift the language barrier that has impeded interspecies communication for millennia?
Microsoft ImageBERT | Cross-modal Pretraining with Large-scale Image-Text Data
A recent paper published by Microsoft researchers proposes a new vision-language pretrained model for image-text joint embedding, ImageBERT, which which achieves SOTA performance on both the MSCOCO and Flickr30k datasets.
Hallo! Hallo! KU Leuven & TU Berlin Introduce ‘RobBERT,’ a SOTA Dutch BERT
A group of researchers from The Katholieke Universiteit Leuven and The Technical University of Berlin recently introduced a Dutch RoBERTa-based language model, RobBERT.
Facebook AI | Text-Based Generation of Interactive Worlds
Researchers recently proposed a new machine learning method for worldbuilding based on content from LIGHT, a research environment open-sourced by Facebook comprising crowd-sourced game locations, characters, and objects, etc.
Google Releases ALBERT V2 & Chinese-Language Models
Google has now released a major V2 ALBERT update and open-sourced Chinese ALBERT models.