AI Technology & Industry Review
In a new paper RL on Incorrect Synthetic Data Scales the Efficiency of LLM Math Reasoning by Eight-Fold, a research team provides insights into how synthetic data affects performance, suggesting that a specific schema can achieve consistent gains over using only positive data, achieving performance by 8× in synthetic data volume.
In a new paper Composable Function-preserving Expansions for Transformer Architectures, a research team from Google DeepMind and University of Toulouse introduces parameter expansion transformations for transformer-based neural networks while preserving functionality, enabling the expansion of the capability of the model as needed.
A Microsoft Research India team presents Varuna, a system for training massive deep learning models on commodity networking that eliminates the need for specialized hyperclusters and alleviates the cost, scale, and resource utilization challenges of deep learning model training.
A DeepMind research team proposes Faster Improvement Rate PBT (FIRE PBT) for Population Based Training (PBT), an automated hyperparameter tuning method for neural network training. The novel approach achieves faster improvement rates and better long-term performance.
Researchers proposed a “radioactive data” technique for subtly marking images in a dataset to help researchers later determine whether they were used to train a particular model.
OneClick.ai, a startup founded by two former Microsoft engineers in Seattle, is on a mission to make AI more accessible to businesses. “We design, build and deploy custom AI models as a scalable API that can be accessed from anywhere. Just prepare your data, and we’ll take care of the rest,” says co-founder and CTO Ning Jiang.