‘Train Large, Then Compress’ – UC Berkeley BAIR Improves Large Transformer Model Training and Inference
Researchers from the Berkeley Artificial Intelligence Research (BAIR) Lab at UC Berkeley explored the effect of Transformer model size on training and inference efficiency.