Amazon’s BERT Optimal Subset: 7.9x Faster & 6.3x Smaller Than BERT
Amazon extracts an optimal subset of architectural parameters for BERT architecture by applying recent breakthroughs in algorithms for neural architecture search.
Tag: BERT
AMBERT: BERT with Multi-Grained Tokenization Achieves SOTA Results on English and Chinese NLU Tasks
AMBERT (A Multigrained BERT) leverages both fine-grained and coarse-grained tokenizations to achieve SOTA performance on English and Chinese language tasks.
How Smart is BERT? Evaluating the Language Model’s Commonsense Knowledge
Researchers dive deep into the large language model to discover how it encodes the structured commonsense knowledge it leverages on downstream commonsense tasks.
Facebook & CMU Introduce TaBERT for Understanding Tabular Data Queries
TaBERT-powered neural semantic parsers showed performance improvements on the challenging benchmark WikiTableQuestions and demonstrated competitive performance on the text-to-SQL dataset Spider.
DeepMind Says Syntactic Biases ‘Helped BERT Do Better’
Researchers add syntactic biases to determine whether and where they can help BERT achieve better understanding.
Google Introduces BLEURT – a BERT-Based NLG Evaluation Metric
Google Research team proposes the automatic metric BLEURT which is based on the highly successful Google language model BERT.
BERTLang Helps Researchers Choose Between BERT Models
Researchers from Bocconi University have prepared an online overview of the commonalities and differences between language-specific BERT models and mBERT.
BERT-of-Theseus: Compressing BERT by Progressive Module Replacing
Researchers propose a novel model compression approach to effectively compress BERT by progressive module replacing.
Up Close and Personal With BERT – Google’s Epoch-Making Language Model
A recent Google Brain paper looks into Google’s hugely successful transformer network — BERT — and how it represents linguistic information internally.
Facebook AI mBART: The Tower of Babel’s Silicon Solution
Facebook AI researchers have further developed the BART model with the introduction of mBART.
Hallo! Hallo! KU Leuven & TU Berlin Introduce ‘RobBERT,’ a SOTA Dutch BERT
A group of researchers from The Katholieke Universiteit Leuven and The Technical University of Berlin recently introduced a Dutch RoBERTa-based language model, RobBERT.
Google Releases ALBERT V2 & Chinese-Language Models
Google has now released a major V2 ALBERT update and open-sourced Chinese ALBERT models.
Voila! SOTA French Language Model ‘CamemBERT’ Debuts
Now, a team from Facebook AI Research, Inria, and Sorbonne Université have released CamemBERT, essentially a French version of Google AI’s game-changing pretrained language model BERT.
Google’s Big Week: Quantum Supremacy, Boosting Search With BERT, Exploring Transfer Learning
Synced Global AI Weekly October 27th
Milestone: BERT Boosts Google Search
In what the company calls “the biggest leap forward in the past five years, and one of the biggest leaps forward in the history of Search,” Google today announced that it has leveraged its pretrained language model BERT to dramatically improve the understanding of search queries.
Huawei’s TinyBERT Is 7X Smaller and 9X Faster Than BERT
Researchers from the Huazhong University of Science and Technology and Huawei Noah’s Ark Lab have introduced TinyBERT, a smaller and faster version of Google’s popular large-scale pre-trained language processing model BERT.
Has BERT Been Cheating? Researchers Say it Exploits ‘Spurious Statistical Cues’
Since Google Research introduced its Bidirectional Transformer (BERT) in 2018 the model has gained unprecedented popularity among researchers. Now, a group of researchers from the National Cheng Kung University Tainan in Taiwan are challenging BERT’s efficacy.
The Staggering Cost of Training SOTA AI Models
While it is exhilarating to see AI researchers pushing the performance of cutting-edge models to new heights, the costs of such processes are also rising at a dizzying rate.
PyTorch Reimplementation of OpenAI GPT-2 Small Model Released
Github developer Hugging Face has updated its repository with a PyTorch reimplementation of the GPT-2 language model small version that OpenAI open-sourced last week, along with pretrained models and fine-tuning examples.