AI Technology & Industry Review
Technical review of the newest machine intelligence research.
In a new paper PokéLLMon: A Human-Parity Agent for Pokémon Battles with Large Language Models, a Georgia Institute of Technology research team introduces PokéLLMon, a pioneering LLM-embodied agent demonstrating human-competent performance in tactical battle games.
In a new paper Learning Universal Predictors, a Google DeepMind research team proposes the utilization of Universal Turing Machines (UTMs) for generating training data, thereby enhancing meta-learning and enabling trained neural networks capable of mastering universal prediction strategies.
In a new paper Distilling Vision-Language Models on Millions of Videos, a research team introduces a straightforward yet highly effective method to adapt image-based vision-language models to video. The approach involves generating high-quality pseudo-captions for millions of videos, outperforming state-of-the-art methods across various video-language benchmarks.
In a new paper Meta-Prompting: Enhancing Language Models with Task-Agnostic Scaffolding, the team introduces meta-prompting. This innovative scaffolding approach proves to be highly effective, surpassing standard prompting by 17.1%, expert (dynamic) prompting by 17.3%, and multi-persona prompting by 15.2%.
In a new paper ChatQA: Building GPT-4 Level Conversational QA Models, an NVIDIA research team introduces ChatQA, a suite of conversational question-answering models that achieve GPT-4 level accuracies without relying on synthetic data from OpenAI GPT models.
In a new paper GATS: Gather-Attend-Scatter, a Google DeepMind research team introduces Gather-Attend-Scatter (GATS), a pioneering module designed to seamlessly combine pretrained foundation models—whether trainable or frozen—into larger multimodal networks.
In a new breakthrough paper Driving and suppressing the human language network using large language models, a research team from Massachusetts Institute of Technology, MIT-IBM Watson AI Lab, University of Minnesota and Harvard University leverages a GPT-based encoding model to identify sentences predicted to elicit specific responses within the human language network.
In a new paper Towards Conversational Diagnostic AI, a research team from Google Research and Google DeepMind introduces AMIE (Articulate Medical Intelligence Explorer), an LLM-based AI system meticulously optimized for clinical history-taking and diagnostic dialogues, showcasing superior diagnostic accuracy and outperforming primary care physicians (PCPs).
Can a collective of moderately-sized LLMs collaboratively constitute a chat AI with equivalent or superior abilities? Motivated by this query, a new paper “Blending Is All You Need: Cheaper, Better Alternative to Trillion-Parameters LLM” confirms this idea and introduces the Blended approach.
In a new paper LangSplat: 3D Language Gaussian Splattin, a research team from Tsinghua University and Harvard University introduces LangSplat, a groundbreaking 3D Gaussian Splatting-based method designed for 3D language fields, which surpasses the state-of-the-art LERF method while boasting a remarkable speed improvement of 199 times.
A research team introduces Gemini, an innovative framework, focusing on both architecture and mapping co-exploration, aims to propel large-scale DNN chiplet accelerators to new heights, achieving an impressive average performance improvement of 1.98× and an energy efficiency boost of 1.41× compared to the state-of-the-art Simba architecture.
In a new paper SOLAR 10.7B: Scaling Large Language Models with Simple yet Effective Depth Up-Scaling, a Upstage AI research team introduces depth up-scaling (DUS), which emerges as an efficient and uncomplicated technique for amplifying LLMs, surpassing existing open-source state-of-the-art LLMs, such as Llama 2 and Mistral 7B.
In a new paper WaveCoder: Widespread And Versatile Enhanced Instruction Tuning with Refined Data Generation, a Microsoft research team introduces CodeOcean, which harnesses source code to explicitly control data quality, significantly improving the generalization ability of fine-tuned LLM models.
In a new paper Wired Perspectives: Multi-View Wire Art Embraces Generative AI, a research team from University of Surrey and Beijing University of Posts and Telecommunications introduces DreamWire, an innovative AI system poised to democratize the creation of MVWA.
A Meta GenAI research team introduces Fairy, a versatile and efficient video-to-video synthesis framework. Fairy stands out for its ability to generate high-quality videos at remarkable speed, producing 120-frame 512×384 videos in just 14 seconds, surpassing previous works by a factor of at least 44×.
In a new paper Autonomous chemical research with large language models, a research team from Carnegie Mellon University and Emerald Cloud Lab introduces an innovative LLMs-Powered system named Coscientist, which autonomously designs, plans, and executes complex scientific experiments, marking a significant leap forward in the integration of laboratory automation technologies with powerful language models.
A Google DeepMind research team introduces a groundbreaking family of multimodal models Gemini, which showcase exceptional proficiency across image, audio, video, and text comprehension, pushing the boundaries of large-scale language modeling, image interpretation, audio processing, and video understanding.
In a recent paper titled “Mathematical Discoveries from Program Search with Large Language Models, a research team introduces FunSearch—a novel approach that elevates LLM-guided evolutionary procedures. FunSearch not only achieves breakthroughs in established open problems but also leads to the discovery of new algorithms.
A recent paper from Tencent’s research team introduces a novel identity-preserving synthesis approach, with a specific focus on human images. The proposed model adopts a direct feed-forward mechanism, eliminating the need for intensive fine-tuning and streamlining the image generation process.
A Microsoft research team introduces TaskWeaver, a cutting-edge, code-first framework designed to empower LLM-powered autonomous agents. TaskWeaver offers a potent and flexible platform for constructing intelligent conversational agents capable of handling complex tasks and seamlessly adapting to domain-specific scenarios.
A collaborative effort between Tencent AI Lab and The University of Sydney introduces GPT4Video, which stands as a unified multi-model framework that endows Large Language Models (LLMs) with the unique ability for both video understanding and generation.
A paper titled “Online Video Quality Enhancement with Spatial-Temporal Look-up Tables” introduces a novel method, STLVQE. This research, conducted by a team from Tongji University and Microsoft Research Asia, pioneers the exploration of the online video quality enhancement problem and presents the first method achieving real-time processing speed.
A research team innovative single-stage category-agnostic diffusion model. This model can generate 3D Neural Radiance Fields (NeRFs) from either text or a single-image input condition through direct model inference, enabling the creation of diverse high-fidelity 3D objects in just 30s/asset.
In a new paper DiLoCo: Distributed Low-Communication Training of Language Models, a Google DeepMind research team presents Distributed Low-Communication (DiLoCo). DiLoCo employs a distributed optimization algorithm that facilitates the training of language models on islands of poorly connected devices, surpassing the performance of fully synchronous models while reducing communication by 500 times.
In a new paper An Embodied Generalist Agent in 3D World, a research team introduces LEO, which stands as an embodied multi-modal and multi-task generalist agent that excels in essential capabilities such as perception, grounding, reasoning, planning, and action within the intricate 3D world.
In a new paper Exponentially Faster Language Modelling, an ETH Zurich research team introduces UltraFastBERT, a variant of the BERT architecture. UltraFastBERT takes a revolutionary approach by replacing feedforward layers with fast feedforward networks, resulting in an impressive 78x speedup over the optimized baseline feedforward implementation.
Microsoft has recently unveiled Orca 2 in a new paper titled “Orca 2: Teaching Small Language Models How to Reason.” to explore how enhanced training signals can augment the reasoning abilities of smaller language models. Notably, Orca 2 surpasses models of similar size, achieving performance levels comparable to or better than models 5-10 times larger.
In a new paper Data Filtering Networks, a research team from Apple and University of Washington introduces the concept of data filtering networks (DFNs). These neural networks, specifically designed for data filtration, demonstrate the capacity to generate extensive, high-quality pre-training datasets efficiently.
In a new paper LRM: Large Reconstruction Model for Single Image to 3D, a research team from Adobe Research and Australian National Univerisity introduces an innovative Large Reconstruction Model (LRM). This groundbreaking model has the remarkable ability to predict a 3D model of an object from a single input image in a mere 5 seconds.
In a new paper E3 TTS: Easy End-to-End Diffusion-based Text to Speech, a Google research team proposes Easy End-to-End Diffusion-based Text to Speech. This streamlined and efficient text-to-speech model hinges solely on diffusion to preserve temporal structure, allowing it to accept plain text as input and generate audio waveforms directly.
An Apple research team presents Large LAnguage model Reinforcement Learning Policy (LLaRP). LLaRP effectively repurposes LLMs for Reinforcement Learning (RL) challenges within the realm of Embodied Artificial Intelligence (AI), achieving a remarkable 1.7 times higher success rate compared to other established baselines and zero-shot LLM applications.
In a new paper Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time, a research team presents DEJAVU, a system that employs a cost-effective algorithm to predict contextual sparsity dynamically for each layer, combined with an asynchronous and hardware-aware implementation to accelerate LLM inference.
A research team from Institute of Science and Technology Austria (ISTA) and Neural Magic Inc. introduces the QMoE framework. This innovative framework offers an effective solution for accurately compressing massive MoEs and conducting swift compressed inference, reducing model sizes by 10–20×, achieving less than 1 bit per parameter.
In a new paper ConvNets Match Vision Transformers at Scale, a Google DeepMind research team challenges the prevailing belief that Vision Transformers possess superior scaling capabilities compared to ConvNets and provides empirical results revealing that ConvNets can indeed hold their own against Vision Transformers at scale.
In a new paper Techniques for Training Consistency Models, an OpenAI research team introduces innovative methods that enable consistency models to learn directly from data, surpassing the performance of consistency distillation (CD) in producing high-quality samples, all while breaking free from the clutches of LPIPS.
In a new paper Large Search Model: Redefining Search Stack in the Era of LLMs, a Microsoft research team presents a novel conceptual framework, large search model, which reimagines the conventional search stack by consolidating various search tasks under a single Large Language Model (LLM).
In a new paper Improving Image Generation with Better Captions, a research team from OpenAI and Microsoft introduces DALL-E 3, a cutting-edge text-to-image generation system that is benchmarked for its prowess in prompt following, coherence, and aesthetics, demonstrating its competitive edge against existing counterparts.
In a new paper SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF, an NVIDIA research team introduces STEERLM, a novel supervised fine-tuning method that empowers end-users to control model responses during inference, surpassing even state-of-the-art baselines, including RLHF models like ChatGPT-3.5.
A Microsoft Azure AI research team introduces “Idea2Img” in their paper, “Idea2Img: Iterative Self-Refinement with GPT-4V(ision) for Automatic Image Design and Generation.”, which leverages the capabilities of GPT-4V(ision) to revolutionize the process of automatic image design and generation with enhanced image quality.
In a new paper MatFormer: Nested Transformer for Elastic Inference, a research team proposes MatFormer, a Transformer architecture that is inherently designed for elasticity, enables the training of a single universal model capable of generating numerous smaller submodels without the need for additional training.