Tag: Sparse Expert Models

AI Machine Learning & Data Science Nature Language Tech Research

Jeff Dean Co-authors Guidelines for Resolving Instability and Quality Issues in the Design of Effective Sparse Expert Models

A Google research team publishes guidelines for designing more practical and reliable sparse expert models. Their pretrained 269B sparse model achieves state-of-the-art results across many natural language processing (NLP) benchmarks.