AI Technology & Industry Review
In a new paper NeMo-Aligner: Scalable Toolkit for Efficient Model Alignment, a team of researchers from Nvidia introduces NeMo-Aligner, a toolkit designed for large-scale LLM model alignment that can efficiently harness the power of hundreds of GPUs for training.
A pioneering research group introduces MovieChat, a novel framework tailored to accommodate extensive video durations exceeding 10,000 frames. This innovative system achieves unprecedented performance in deciphering prolonged video content.
In a new paper TriForce: Lossless Acceleration of Long Sequence Generation with Hierarchical Speculative Decoding, a research team from CMU and Meta introduces TriForce—a hierarchical speculative decoding system tailored for scalable long sequence generation, reaching up to 2.31× on an A100 GPU.
In a new paper NExT: Teaching Large Language Models to Reason about Code Execution, a Google DeepMind research team proposes Naturalized Execution Tuning (NExT), a method aims to equip LLMs with the ability to scrutinize program execution traces and deduce runtime behaviors through chain-of-thought (CoT) rationales.
An NVIDIA research team presents ScaleFold, a novel and scalable training methodology tailored for the AlphaFold model, which accomplishes the OpenFold partial training task in a mere 7.51 minutes—over six times faster than the benchmark baseline—ultimately slashing the AlphaFold’s initial training time to a remarkable 10 hours.
A Google DeepMind research team introduce RecurrentGemma, an open language model built on Google’s innovative Griffin architecture, which reduces memory usage and facilitates efficient inference on lengthy sequences, thereby unlocking new possibilities for highly efficient small language models in environments where resources are limited.
A Google research team unveils the latest iteration of MobileNets: MobileNetV4 (MNv4). This cutting-edge model boasts an impressive 87% ImageNet-1K accuracy, coupled with an astonishingly low Pixel 8 EdgeTPU runtime of merely 3.8ms.
In a new paper LM Transparency Tool: Interactive Tool for Analyzing Transformer Language Models, a research team from Meta, University College London and Universitat Politècnica de Catalunya introduces the LM Transparency Tool (LM-TT), an open-source interactive toolkit designed for dissecting Transformer-based language models.
In a new paper Streaming Dense Video Captioning, a Google research team proposes a streaming dense video captioning model, which revolutionizes dense video captioning by enabling the processing of videos of any length and making predictions before the entire video is fully analyzed, thus marking a significant advancement in the field.
A collaborative effort involving researchers from 33 institutions presents AURORA-M, the inaugural open-source model not only excels in multilingual understanding and coding tasks but also underscores the collaborative ethos of the open-source community, promoting transparency and accessibility in AI development.
A research team from Huawei and Peking University introduces DiJiang, a groundbreaking Frequency Domain Kernelization approach, which facilitates the transition to a linear complexity model with minimal training overhead, achieving performance akin to LLaMA2-7B across various benchmarks, but at just 1/50th of the training cost.
A King’s College London research team delves into a theoretical exploration of the transformer architecture, employing the lens of topos theory. This innovative approach conjectures that the factorization through “choose” and “eval” morphisms can yield effective neural network architecture designs.
A research team from University of California, Berkeley presents a causal transformer model trained via autoregressive prediction of sensorimotor trajectories, culminating in the remarkable feat of enabling a full-sized humanoid to navigate the streets of San Francisco in a zero-shot manner.
In a new paper Stealing Part of a Production Language Model, a research team introduces the first model-stealing attack that unveils precise, nontrivial information from black-box production language models such as OpenAI’s ChatGPT or Google’s PaLM-2.
A research team from Google DeepMind and University of British Columbia presents Genie, the first generative interactive environment capable of seamlessly generating a diverse array of controllable virtual worlds based on textual prompts, synthetic images, photographs, and even sketches.
A ByteDance research team presents AnimateDiff-Lightning, a novel approach that utilizes progressive adversarial diffusion distillation, catapulting video generation into a realm of lightning-fast performance while simultaneously achieving unprecedented results in few-step video generation.
In a new paper VideoAgent: Long-form Video Understanding with Large Language Model as Agent, a Stanford University research team introduces VideoAgent, an innovative approach simulates human comprehension of long-form videos through an agent-based system, showcasing superior effectiveness and efficiency compared to current state-of-the-art methods.
Google introduces Gemma, a suite of lightweight, cutting-edge open models derived from the same research and technology underpinning the powerful Gemini models, which mark a significant leap forward in performance relative to existing open models across academic benchmarks for language comprehension, reasoning, and safety.
In a new paper Behavior Generation with Latent Actions, a research team introduces the Vector-Quantized Behavior Transformer (VQ-BeT), an innovative model offers a solution for behavior generation, addressing multimodal action prediction, conditional generation, and partial observations.
In a new paper BasedAI: A decentralized P2P network for Zero Knowledge Large Language Models (ZK-LLMs), Based Labs proposes BasedAI, which offers a decentralized approach that seamlessly integrates FHE with LLMs to uphold data confidentiality without sacrificing performance.
In a new paper Beyond Language Models: Byte Models are Digital World Simulators, a research team introduces bGPT, a pioneering model engineered explicitly for processing binary data and simulating the digital world through next-byte prediction.
In a new paper Nemotron-4 15B Technical Report , an NVIDIA research team introduces Nemotron-4 15B. Nemotron-4 15B is comprising 15 billion parameters, is trained on an extensive corpus of 8 trillion text tokens, showcasing unparalleled multilingual capabilities among models of comparable size.
In a new paper LongRoPE: Extending LLM Context Window Beyond 2 Million Tokens, a Microsoft research team introduces LongRoPE, a pioneering method that extends the context window of pre-trained LLMs to an impressive 2048k tokens while preserving performance at the original short context window.
In a new paper Learning by Reconstruction Produces Uninformative Features For Perception, researchers Randall Balestriero and Yann LeCun shed light on why reconstruction-based learning yields compelling reconstructed samples but falters in delivering competitive latent representations for perception.
An Apple research team introduces Keyframer, a groundbreaking animation prototyping tool fueled by LLM technology. Keyframer facilitates the generation of animations from static images (SVGs), empowering users to explore design alternatives, facilitate comparisons, and foster ideation.
In a recent technical report, OpenAI introduces Sora, a groundbreaking text-to-video model. Sora stands out for its ability to generate videos and images spanning a wide range of durations, aspect ratios, and resolutions, producing up to a minute of high-definition video content.
A research team from Google DeepMind and Stanford University introduces a groundbreaking algorithm known as universal neural functionals (UNFs), which autonomously constructs permutation-equivariant models for any weight space, offering a versatile solution to the architectural constraints encountered in prior works.
An NVIDIA research team introduces Audio Flamingo, a groundbreaking audio language model that incorporates in-context learning (ICL), retrieval augmented generation (RAG), and multi-turn dialogue capabilities, achieving SOTA performance across various audio understanding tasks.
In a new paper Nomic Embed: Training a Reproducible Long Context Text Embedder, a Nomic AI research team introduces nomic-embed-text-v1, which marks the inception of the first fully reproducible, open-source, open-weights, open-data text embedding model, capable of handling an extensive context length of 8192 in English.
In a new paper PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large Language Models, a Georgia Institute of Technology research team introduces PokéLLMon, a pioneering LLM-embodied agent demonstrating human-competent performance in tactical battle games.
In a new paper Learning Universal Predictors, a Google DeepMind research team proposes the utilization of Universal Turing Machines (UTMs) for generating training data, thereby enhancing meta-learning and enabling trained neural networks capable of mastering universal prediction strategies.
In a new paper Distilling Vision-Language Models on Millions of Videos, a research team introduces a straightforward yet highly effective method to adapt image-based vision-language models to video. The approach involves generating high-quality pseudo-captions for millions of videos, outperforming state-of-the-art methods across various video-language benchmarks.
In a new paper Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding, the team introduces meta-prompting. This innovative scaffolding approach proves to be highly effective, surpassing standard prompting by 17.1%, expert (dynamic) prompting by 17.3%, and multi-persona prompting by 15.2%.
In a new paper ChatQA: Building GPT-4 Level Conversational QA Models, an NVIDIA research team introduces ChatQA, a suite of conversational question-answering models that achieve GPT-4 level accuracies without relying on synthetic data from OpenAI GPT models.
In a new paper GATS: Gather-Attend-Scatter, a Google DeepMind research team introduces Gather-Attend-Scatter (GATS), a pioneering module designed to seamlessly combine pretrained foundation models—whether trainable or frozen—into larger multimodal networks.
In a new breakthrough paper Driving and suppressing the human language network using large language models, a research team from Massachusetts Institute of Technology, MIT-IBM Watson AI Lab, University of Minnesota and Harvard University leverages a GPT-based encoding model to identify sentences predicted to elicit specific responses within the human language network.
In a new paper Towards Conversational Diagnostic AI, a research team from Google Research and Google DeepMind introduces AMIE (Articulate Medical Intelligence Explorer), an LLM-based AI system meticulously optimized for clinical history-taking and diagnostic dialogues, showcasing superior diagnostic accuracy and outperforming primary care physicians (PCPs).
Can a collective of moderately-sized LLMs collaboratively constitute a chat AI with equivalent or superior abilities? Motivated by this query, a new paper “Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM” confirms this idea and introduces the Blended approach.
In a new paper LangSplat: 3D Language Gaussian Splattin, a research team from Tsinghua University and Harvard University introduces LangSplat, a groundbreaking 3D Gaussian Splatting-based method designed for 3D language fields, which surpasses the state-of-the-art LERF method while boasting a remarkable speed improvement of 199 times.
A research team introduces Gemini, an innovative framework, focusing on both architecture and mapping co-exploration, aims to propel large-scale DNN chiplet accelerators to new heights, achieving an impressive average performance improvement of 1.98× and an energy efficiency boost of 1.41× compared to the state-of-the-art Simba architecture.