Tag: GELU

AI Machine Learning & Data Science Research

Gaussian Error Linear Unit Activates Neural Networks Beyond ReLU

Results of the various experiments show GELU consistently has the best performance compared with ReLU and ELU, and can be considered a viable alternative to previous nonlinear approaches.