Google Trains a 540B Parameter Language Model With Pathways, Achieving ‘Breakthrough Performance’
A Google Research team further explores the scaling approach for improving language modelling, leveraging the new Pathways distributed ML system to train a 540 billion parameter autoregressive transformer, Pathways Language Model (PaLM), that achieves state-of-the-art few-shot performance.