Tag: optimization strategy

AI Machine Learning & Data Science Research

New IEEE Research Equips Gradient Descent with Angular Information to Boost DNN Training

An IEEE team proposes AngularGrad — a novel optimization algorithm that takes both gradient direction and angular information into consideration. The method successfully reduces the zig-zag effect in the optimization trajectory and speeds up convergence.