Uber ATG Open-Sources Neuropod DL Inference Engine

Rideshare giant Uber continues to explore deep learning’s potential to provide safer and more reliable self-driving technologies. This week, the Uber Advanced Technologies Group (ATG) released Neuropod, an open-source library that provides a uniform interface for running deep learning (DL) models from multiple frameworks in C++ and Python.

“As our self-driving software evolves, we are always looking for new ways to improve our models and sometimes that means experimenting with different deep learning frameworks,” writes Uber ATG Senior Autonomy Engineer Vivek Panyam in a company blog.

Over the past few years, Uber ATG researchers have used DL frameworks such as Caffe2, TensorFlow, PyTorch, and TorchScript, and they are also looking to try out newly open-sourced frameworks such as JAX, Trax, Flax, Haiku, and RLax.

Although experimenting with new frameworks is fairly simple, actually adding production support for new DL frameworks across all systems and processes is very time-consuming, and Uber developed Neuropod to streamline that process. Neuropod functions as an abstraction layer on top of existing deep learning frameworks, providing a uniform interface for running different DL models.

Every Neuropod model implements a problem definition — a formal description of a problem for models to solve. As a result, any models that solve the same problem are interchangeable even if they use different frameworks. Existing models can be wrapped in a Neuropod package, which contains the original model along with metadata, test data, and custom ops if any.

Since its internal release in early 2019, hundred of Neuropod models have been deployed across Uber ATG, Uber AI, and the core Uber business — including models for demand forecasting, estimated time of arrival (ETA) prediction for rides, menu transcription for Uber Eats, and object detection models for self-driving vehicles.

Neuropod makes it easy for researchers to build models in a framework of their choosing while also simplifying product-ionization of these models, says the company. Neuropod currently supports TensorFlow, PyTorch, TorchScript, and Keras.

Journalist: Yuan Yuan | Editor: Micahel Sarazen