transfer learning

by Synced 2023-04-20 0

Huawei’s DiffFit Unlocks the Transferability of Large Diffusion Models to New Domains

In the new paper DiffFit: Unlocking Transferability of Large Diffusion Models via Simple Parameter-Efficient Fine-Tuning, a Huawei Noah’s Ark Lab research team introduces DiffFit, a parameter-efficient fine-tuning technique that enables fast adaptation to new domains for diffusion image generation. Compared to full fine-tuning approaches, DiffFit achieves 2x training speed-ups while using only ~0.12 percent of trainable parameters.

by Synced 2022-09-27 0

AI Machine Learning & Data Science Research

Google’s Promptagator Creates Task-Specific Neural Retrievers From Only 8 Examples

In the new paper Promptagator: Few-shot Dense Retrieval From 8 Examples, a Google Research team proposes Prompt-based Query Generation for Retriever (Promptagator), a novel and simple approach for few-shot retrieval that leverages large language model (LLM) prompting to generate synthetic task-specific training data.

by Synced 2022-09-14 0

AI Machine Learning & Data Science Research

DeepMind’s Model-Based Offline Options Framework Supports Automatic Skill & Behaviour Discovery, Boosts Transfer Capabilities

In the new paper MO2: Model-Based Offline Options, a DeepMind research team introduces Model-Based Offline Options (MO2), an offline hindsight bottleneck options framework that supports sample-efficient option discovery over continuous state-action spaces for efficient skill transfer to new tasks.

by Synced 2022-07-06 1

AI Machine Learning & Data Science Research

NYU Explores the Principles for Modelling Neural Collapse and Its Role in Generalization

In the new paper Neural Collapse: A Review on Modelling Principles and Generalization, researchers from New York University analyze Neural Collapse (NC) and present a thought model to explain the effects of variance collapse, aiming at a better understanding of the generalization capabilities of neural networks.

by Synced 2021-05-18 2

AI Machine Learning & Data Science Research

Facebook Transfer Learning Method Boosts Code Autocompletion Accuracy by Over 50%

A research team from Facebook shows how the power of transfer learning can enable pretraining on non-IDE, non-autocompletion and different-language example code sequences before fine-tuning on the autocompletion prediction task to improve model accuracy by over 50 percent on very small fine-tuning datasets and over 10 percent on 50k labelled examples.

by Synced 2021-03-18 2

AI Machine Learning & Data Science Research

Increasing Data Efficiency: Hugging Face Quantifies the Benefits of Prompts for Pretrained Language Models

A research team from Hugging Face shows that prompting is indeed beneficial for fine-tuning pretrained language models, and that this benefit can be quantified as worth hundreds of data points on average across classification tasks.

by Synced 2021-02-03 1

AI Machine Learning & Data Science Nature Language Tech Research

New Transfer Learning Approach Summarizes Historical Texts in Modern Languages

Researchers from the University of Sheffield, Beihang University, and Open University’s Knowledge Media Institute have proposed a transfer learning approach that can automatically process historical texts at a semantic level to generate modern language summaries.

by Synced 2020-07-23 1

AI Machine Learning & Data Science Research

Microsoft & MIT Apply Adversarially Robust Models for Better Transfer Learning

Despite being less accurate on ImageNet, adversarially robust neural networks can match or improve their standard counterparts’ transfer performance.

by Synced 2019-11-07 1

AI Research

Google T5 Explores the Limits of Transfer Learning

Synced invited Samuel R. Bowman, an Assistant Professor at New York University who works on artificial neural network models for natural language understanding, to share his thoughts on the “Text-to-Text Transfer Transformer” (T5) framework.

by Synced 2019-09-20 2

AI Conference Industry

RE•WORK AI in Finance | Federated AI, Reinforcement Learning and BERT

At last week’s Re•Work AI in Finance Conference in New York, researchers and engineers from banks and academia alike shared their thoughts on current AI research and applications in the finance world.

by Synced 2019-01-18 1

AI Research

Automatic Creation of Machine Learning Algorithms for Autonomous Vehicles

To make ML-based solutions available for a wider variety of deployment scenarios, Waymo’s autonomous driving team has collaborated with Google AI Brain Team researchers on a system that automates the creation of high quality and low latency neural networks on existing AutoML architectures.

by Synced 2018-06-19 0

AI Conference United States

CVPR 2018 Kicks Off; Best Papers Announced

The 2018 Conference on Computer Vision and Pattern Recognition (CVPR) opened today in Salt Lake City, USA. The CVPR organizing committee used the occasion to announce its coveted Best Paper and Best Student Paper selections.

by Synced 2017-06-30 0

Research

Understanding the Mechanisms of Deep Transfer Learning for Medical Images

This paper systematically studies how a convolutional neural network, trained on ImageNet for image classification tasks, works on medical images – or more precisely on ultrasound images – for the “Kidney Detection” problem.