Facebook AI Proposes Group Normalization Alternative to Batch Normalization
As Facebook struggles with fallout from the Cambridge Analytica scandal, its research arm today delivered a welcome bit of good news in deep learning. Research Engineer Dr. Yuxin Wu and Research Scientist Dr. Kaiming He proposed a new Group Normalization (GN) technique they say can accelerate deep neural network training with small batch sizes.