Google Releases ALBERT V2 & Chinese-Language Models
Google has now released a major V2 ALBERT update and open-sourced Chinese ALBERT models.
Tag: BERT
Voila! SOTA French Language Model ‘CamemBERT’ Debuts
Now, a team from Facebook AI Research, Inria, and Sorbonne Université have released CamemBERT, essentially a French version of Google AI’s game-changing pretrained language model BERT.
Google’s Big Week: Quantum Supremacy, Boosting Search With BERT, Exploring Transfer Learning
Synced Global AI Weekly October 27th
Milestone: BERT Boosts Google Search
In what the company calls “the biggest leap forward in the past five years, and one of the biggest leaps forward in the history of Search,” Google today announced that it has leveraged its pretrained language model BERT to dramatically improve the understanding of search queries.
Huawei’s TinyBERT Is 7X Smaller and 9X Faster Than BERT
Researchers from the Huazhong University of Science and Technology and Huawei Noah’s Ark Lab have introduced TinyBERT, a smaller and faster version of Google’s popular large-scale pre-trained language processing model BERT.
Has BERT Been Cheating? Researchers Say it Exploits ‘Spurious Statistical Cues’
Since Google Research introduced its Bidirectional Transformer (BERT) in 2018 the model has gained unprecedented popularity among researchers. Now, a group of researchers from the National Cheng Kung University Tainan in Taiwan are challenging BERT’s efficacy.
The Staggering Cost of Training SOTA AI Models
While it is exhilarating to see AI researchers pushing the performance of cutting-edge models to new heights, the costs of such processes are also rising at a dizzying rate.
PyTorch Reimplementation of OpenAI GPT-2 Small Model Released
Github developer Hugging Face has updated its repository with a PyTorch reimplementation of the GPT-2 language model small version that OpenAI open-sourced last week, along with pretrained models and fine-tuning examples.