optimization strategy

by Synced 2021-05-28 2

New IEEE Research Equips Gradient Descent with Angular Information to Boost DNN Training

An IEEE team proposes AngularGrad — a novel optimization algorithm that takes both gradient direction and angular information into consideration. The method successfully reduces the zig-zag effect in the optimization trajectory and speeds up convergence.

by Synced 2021-05-10 2

AI Machine Learning & Data Science Research

Imperial College London Proposes Optimal Training of Variational Quantum Algorithms Without Barren Plateaus

Imperial College London researchers show how to optimally train a variational quantum algorithm to represent quantum states and propose a stable variant of the quantum natural gradient, a generalized quantum natural gradient that can be trained free of barren plateaus.

by Synced 2017-04-29 0

Research

How to Train a Very Large and Deep Model on One GPU?

The article discusses about the limitation of using existing GPU to train neural network and how one can improve to train larger model.