New IEEE Research Equips Gradient Descent with Angular Information to Boost DNN Training
An IEEE team proposes AngularGrad — a novel optimization algorithm that takes both gradient direction and angular information into consideration. The method successfully reduces the zig-zag effect in the optimization trajectory and speeds up convergence.