AI Technology & Industry Review
In a new paper Large Language Models as Optimizers, a Google DeepMind research team introduces Optimization by PROmpting (OPRO), an effective method that leverages large language models (LLMs) as optimizers, which can generate optimization solutions conditioned on the natural language that describes the optimization task.
An IEEE team proposes AngularGrad — a novel optimization algorithm that takes both gradient direction and angular information into consideration. The method successfully reduces the zig-zag effect in the optimization trajectory and speeds up convergence.
Imperial College London researchers show how to optimally train a variational quantum algorithm to represent quantum states and propose a stable variant of the quantum natural gradient, a generalized quantum natural gradient that can be trained free of barren plateaus.
The article discusses about the limitation of using existing GPU to train neural network and how one can improve to train larger model.