Facebook Open-Sources PySlowFast Codebase for Video Understanding

Facebook AI Research (FAIR) has been contributing heavily to video understanding research in recent years. At October’s ICCV 2019 the team unveiled a Python-based codebase, PySlowFast. FAIR as now open-sourced PySlowFast, along with a pretrained model library and a pledge to continue adding cutting-edge resources to the project.

The name “PySlowFast” derives from a novel duality — the model has both a slow pathway that operates at a low frame rate to capture spatial semantics, and a lightweight, fast pathway that operates at a high frame rate, captures motion at fine temporal resolution, and can learn useful temporal information for video recognition.

The introduction of PySlowFast addresses a couple of needs for ML researchers. First, there was no concise, efficient and easy-to-modify video understanding codebase in the open source community. Second, recreating today’s state-of-the-art deep learning models can be a headache, as such models often require tens of GFlops, several days of training, and repeated experimental tuning to make every detail correct. This can be prohibitively time- and resource-consuming for many researchers.

PySlowFast will enable researchers to easily reproduce video classification and action detection algorithms, whether they are basic or cutting-edge. FAIR has also open-sourced a number of pretrained models to save researchers the trouble of repeatedly training sessions.

PySlowFast also includes a dedicated interface to support multi-modal video understanding, video self-supervised learning and other tasks through simple editing. FAIR says PySlowFast will be actively updated with cutting-edge algorithms in real time to ensure it remains a current and reliable benchmark in the field of video understanding.

After completing the installation, users can test performance on different video databases by downloading the pretrained model and corresponding configuration file provided by MODEL_ZOO and running the following code:

python tools/run_net.py \
--cfg configs/Kinetics/C2D_8x8_R50.yaml \
DATA.PATH_TO_DATA_DIR path_to_your_dataset \
NUM_GPUS 2 \

The PySlowFast Codebase is on GitHub. The related papers are on arXiv: SlowFast Networks for Video Recognition and Non-local Neural Networks.

---

Author: Reina Qi Wan | Editor: Michael Sarazen