In recent developments within the realm of large-scale models, a tantalizing prospect has emerged: the potential to enhance pre-training efficacy by scaling up data, model size, and training duration. This augmentation appears to yield promising results across a wide spectrum of downstream tasks. Furthermore, these all-encompassing models possess the capacity to surpass their narrowly specialized counterparts, which are trained on task-specific datasets.
Nevertheless, the application of this transformative concept encounters significant hurdles within the field of robotics. Robotics, by its nature, grapples with the scarcity and narrow focus of datasets pertaining to robotic interactions. These datasets frequently exhibit constraints along various dimensions, concentrating on singular environments, specific sets of objects, or a limited range of tasks.
To surmount these challenges, DeepMind, in collaboration with 33 academic laboratories, introduces a groundbreaking paper titled “Open X-Embodiment: Robotic Learning Datasets and RT-X Models.” This paper heralds the arrival of RT-1-X, a novel robotics transformer (RT) model that evolves from RT-1. RT-1-X is meticulously trained on the novel Open X-Embodiment dataset constructed by the researchers. Remarkably, this model showcases a remarkable 50% improvement in success rates compared to methods developed in isolation for individual robots.
The primary objectives of this undertaking can be summarized as follows:
- Demonstrating Positive Transfer: The first aim of this research is to showcase that policies crafted from a diverse array of robotic data and environments enjoy the benefits of positive transfer. These policies exhibit superior performance when compared to those trained exclusively on data from specific evaluation setups.
- Facilitating Future Research: The second goal is to contribute datasets, data formats, and models to the robotics community, thereby empowering and encouraging future research endeavors focusing on X-embodiment models.
To realize the first goal, the research team illustrates that several contemporary robotic learning methodologies can seamlessly harness the X-embodiment data with minimal adjustments. In particular, they train the RT-1 and RT-2 models on nine distinct robotic manipulators. The resulting models, collectively referred to as RT-X, surpass the capabilities of policies trained solely on data derived from the evaluation domain. These models exhibit superior generalization and innovative capabilities.
Addressing the second goal, the researchers unveil the Open X-Embodiment (OXE) Repository, a comprehensive resource comprising a dataset featuring 22 diverse robotic embodiments from 21 different institutions. This repository empowers the robotics community to embark on further research into X-embodiment models. Furthermore, the team provides open-source tools designed to streamline research efforts in this domain.
Empirical studies underscore the transformative potential of Transformer-based policies trained on the constructed dataset. These policies demonstrate remarkable positive transfer between the various robots encompassed within the dataset. Notably, the RT-1-X policy exhibits a 50% higher success rate compared to state-of-the-art methods. Meanwhile, the more expansive vision-language-model-based iteration, RT-2-X, showcases approximately threefold improvements in generalization over models exclusively trained on evaluation embodiment data.
In addition to these significant advancements, the research team offers multiple invaluable resources to the robotics community. These include a unified X-robot and X-institution dataset, sample code elucidating data utilization, and the RT-1-X model, which serves as a foundational stepping stone for future explorations.
In conclusion, this collaborative effort between DeepMind and 33 academic labs not only validates the feasibility and practicality of X-robot learning but also equips researchers with the tools needed to propel research in this direction. The researchers aspire for this work to serve as both an inspiring example and a catalyst for further advancements in the field of robotic learning.
The paper Open X-Embodiment: Robotic Learning Datasets and RT-X Models on github.io.
Author: Hecate He | Editor: Chain Zhang
We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.