Model Distillation

by Synced 2024-01-29 1

Google and UT Austin’s Game-Changing Approach Distills Vision-Language Models on Millions of Videos

In a new paper Distilling Vision-Language Models on Millions of Videos, a research team introduces a straightforward yet highly effective method to adapt image-based vision-language models to video. The approach involves generating high-quality pseudo-captions for millions of videos, outperforming state-of-the-art methods across various video-language benchmarks.

by Synced 2022-01-26 3

AI Machine Learning & Data Science Research

AutoDistill: An End-to-End Fully Automated Distillation Framework for Hardware-Efficient Large-Scale NLP Models

University of Illinois Urbana-Champaign and Google researchers introduce AutoDistill, an end-to-end fully automated model distillation framework that integrates model architecture exploration and multi-objective optimization for building hardware-efficient pretrained natural language processing models.

by Synced 2021-04-28 3

AI Machine Learning & Data Science Research

Google’s 1.3 MiB On-Device Model Brings High-Performance Disfluency Detection Down to Size

A research team from Google Research proposes small, fast, on-device disfluency detection models based on the BERT architecture. The smallest model size is only 1.3 MiB, representing a size reduction of two orders of magnitude and an inference latency reduction of a factor of eight compared to state-of-the-art BERT-based models.