In a bid to provide an easier path for researchers and engineers seeking to adopt differential privacy (DP) in machine learning (ML) and help accelerate DP research in the field, Facebook AI this week released a new high-speed library called Opacus.
The ML community in recent years has seen growing interest in differential privacy, which is a mathematically rigorous framework often used in analytics for quantifying the anonymization of sensitive data. The goal of Opacus is to preserve the privacy of each training sample while limiting any negative impact on the final model’s accuracy.
Opacus is designed for training PyTorch models with DP in a manner that’s more scalable than existing state-of-the-art methods, the Facebook researchers explain in a blog post. Opacus achieves this by modifying a standard PyTorch optimizer to measure and enforce DP during training.
ML datasets are often crowdsourced and may contain sensitive information. Their use therefore requires techniques that meet the demands of the applications while also providing principled and rigorous privacy guarantees.
Opacus offers speed and safety — it can compute batched per-sample gradients and is processed at high speed on the GPU for an entire batch of parameters. It’s also flexible, enabling engineers and researchers to quickly prototype their ideas by mixing and matching their code with PyTorch code and pure Python code.
The researchers say Opacus defines a lightweight API due to a novel PrivacyEngine abstraction that can track users’ privacy budgets at any given point while also working on model gradients. After training, the resulting artifact is a standard PyTorch model with no extra steps or hurdles for deploying private models.
The library also includes pretrained and fine-tuned models, tutorials for large-scale models, and an infrastructure specifically designed for experiments in privacy research.
The team believes that as ML applications and research continue to accelerate, it is important that ML researchers have access to simple tools that can provide mathematically rigorous privacy guarantees without slowing down the training process.
By developing PyTorch tools like Opacus, the Facebook researchers hope to democratize privacy-preserving resources and bridge the divide between the security community and general ML engineers with a faster, more flexible platform using PyTorch.
The Opacus library has been open-sourced on GitHub.
Reporter: Yuan Yuan | Editor: Michael Sarazen
Synced Report | A Survey of China’s Artificial Intelligence Solutions in Response to the COVID-19 Pandemic — 87 Case Studies from 700+ AI Vendors
This report offers a look at how China has leveraged artificial intelligence technologies in the battle against COVID-19. It is also available on Amazon Kindle. Along with this report, we also introduced a database covering additional 1428 artificial intelligence solutions from 12 pandemic scenarios.
Click here to find more reports from us.
We know you don’t want to miss any story. Subscribe to our popular Synced Global AI Weekly to get weekly AI updates.
0 comments on “High-Speed Privacy Protection: Facebook Opacus Trains PyTorch Models With DP”