In the new paper DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning, a Huawei Noah’s Ark Lab research team introduces DiffFit, a parameter-efficient fine-tuning technique that enables fast adaptation to new domains for diffusion image generation. Compared to full fine-tuning approaches, DiffFit achieves 2x training speed-ups while using only ~0.12 percent of trainable parameters.
In the new paper Promptagator: Few-shot Dense Retrieval From 8 Examples, a Google Research team proposes Prompt-based Query Generation for Retriever (Promptagator), a novel and simple approach for few-shot retrieval that leverages large language model (LLM) prompting to generate synthetic task-specific training data.
In the new paper MO2: Model-Based Offline Options, a DeepMind research team introduces Model-Based Offline Options (MO2), an offline hindsight bottleneck options framework that supports sample-efficient option discovery over continuous state-action spaces for efficient skill transfer to new tasks.
In the new paper Neural Collapse: A Review on Modelling Principles and Generalization, researchers from New York University analyze Neural Collapse (NC) and present a thought model to explain the effects of variance collapse, aiming at a better understanding of the generalization capabilities of neural networks.
A research team from Facebook shows how the power of transfer learning can enable pretraining on non-IDE, non-autocompletion and different-language example code sequences before fine-tuning on the autocompletion prediction task to improve model accuracy by over 50 percent on very small fine-tuning datasets and over 10 percent on 50k labelled examples.
To make ML-based solutions available for a wider variety of deployment scenarios, Waymo’s autonomous driving team has collaborated with Google AI Brain Team researchers on a system that automates the creation of high quality and low latency neural networks on existing AutoML architectures.