AI Technology & Industry Review
In a new paper How to Boost Any Loss Function, a Google research team provides a constructive, formal answer, demonstrating that any loss function can be optimized with boosting.
In a new paper TextGrad: Automatic ‘Differentiation’ via Text, a research team from Stanford University and CZ Biohub introduces TEXTGRAD, a robust framework that performs automatic differentiation through text. In this system, LLMs generate comprehensive, natural language suggestions to optimize variables in computation graphs.
MegEngine Teams propose large-kernel convolution optimization strategy, speeding up 31*31 convolutional neural networks 10 times.
A research team from Hugging Face introduces a block pruning approach targeting both small and fast models, which learns to eliminate full components of the original model while effectively dropping a large number of attention heads.
Researchers from Carnegie Mellon University, the University of Texas at Austin and Facebook AI propose a novel paradigm to optimize widths for each CNN layer. The method is compatible across various width optimization algorithms and networks and achieves up to a 320x reduction in width optimization overhead without compromising top-1 accuracy on ImageNet.