AI Technology & Industry Review
In a new paper Prodigy: An Expeditiously Adaptive Parameter-Free Learner, a research team from Samsung AI Center and Meta AI presents two novel modifications, Prodigy and Resetting, to enhance the D-Adaptation method’s worst-case non-asymptotic convergence rate, achieving faster convergence rates and better optimization outputs.
In the new paper Automatic Prompt Optimization with “Gradient Descent” and Beam Search, a Microsoft research team presents Automatic Prompt Optimization, a simple and general prompt optimization algorithm that automatically improves prompts for large language models, significantly reducing the time and energy spent on manual prompting approaches.
In the new paper Optimistic Meta-Gradients, a DeepMind research team explores the connection between gradient-based meta-learning and convex optimization, demonstrating that optimism in meta-learning is achievable via the Bootstrapped Meta-Gradients approach.
In the new paper Transformer-Based Learned Optimization, a Google Research and Lund University team presents Optimus, an expressive neural network architecture for learned optimization that captures complex dependencies in the parameter space and achieves competitive results on real-world tasks and benchmark optimization problems.
A research team from Google Research and UC Berkeley proposes PRIME, an offline data-driven approach that can architect hardware accelerators without any form of simulations. Compared to state-of-the-art simulation-driven methods, PRIME achieves impressive performance improvements of up to 1.54× while reducing the total required simulation time by up to 99 percent.
A research team from the University of California, Princeton University and ETH Zurich proposes RLQP, an accelerated QP solver based on operator-splitting QP (OSQP) that uses deep reinforcement learning (RL) to speed up the solver’s convergence rate.
A research team from Google Research combines the benefits of implicit differentiation and autodiff and proposes a unified, efficient and modular approach for implicit differentiation of optimization problems.
Probability theory is a mathematical framework for quantifying our uncertainty about the world, and is a fundamental building block in the study of machine learning. The purpose of this article is to provide the vocabulary and mathematics needed before applying probability theory to machine learning tasks.
This is the second in a special Synced series of introductory articles on traditionally theoretical fields of studies and their impact on modern-day machine learning.
To apply AI technology to the oil and gas industry, oil companies and startups generally first establish either a research group or a research center for the purpose. AI for oil and gas is a huge potential market, expected to reach US$2.85 billion by 2022.
RECOS model supply us with a signal analysis perspective of Convolution Neural Network. From this perspective, we can see the effectiveness of activation and deep architectures.