AI Technology & Industry Review
In a new paper ConvNets Match Vision Transformers at Scale, a Google DeepMind research team challenges the prevailing belief that Vision Transformers possess superior scaling capabilities compared to ConvNets and provides empirical results revealing that ConvNets can indeed hold their own against Vision Transformers at scale.
In the new paper Designing BERT for Convolutional Networks: Sparse and Hierarchical Masked Modeling, a research team from Peking University, ByteDance, and the University of Oxford presents Sparse Masked Modelling with Hierarchy (SparK), the first BERT-style pretraining approach that can be used on convolutional models without any backbone modifications.
A Google Brain research team explores the internal representation structures of ViTs and CNNs on image classification tasks, providing insights on key differences between the two approaches.
This method, with considerable computation time and controllable model size, is particularly simple and very useful for implementations on mobile devices such as phones, drones, etc.
This repository did some toy experiments based on Tensorflow in order to introduce some deep learning concepts which are used for image recognition and language modeling.
Softmax loss has become a standard build-in loss function for a lot of mathematical tools like TensorFlow, Torch and Caffe. It is mainly used for classification and has its advantages and disadvantages, the latter of which is the focus of this paper.
Google’s DeepMind introduced its Relation Network (RN) module and a Visual Interaction Network (VIN) that achieved superhuman performance in specific tests by employing a different type of reasoning concerning objects and physical interactions.
Researchers from Microsoft Research Asia and Shanghai Jiaotong University have recently published a paper on transferring visual attributes of images using a new technique called Deep Image Analogy.
This is a comprehensive guide on how to calculate and visualize the receptive field information of a convolutional neural network.
“We want to be able to train a machine learning model, like Google Translate, in a few hours,” said Prof. Hwu. Best known for enabling the parallelism capability of the GPU to solve scientific problems like AI.