Apple Reveals Design of Its On-Device ML System for Federated Evaluation and Tuning

Apple has revealed system design details for its federated task processing system, demonstrating the potential of federated evaluation and tuning for on-device ML system personalization.

Federated Learning (FL) enables model training on a large corpus of decentralized data, a de-identification approach that addresses public concerns and other issues regarding data privacy, ownership and locality. FL is seeing significant interest from both research and application perspectives, and its potential deployment on end devices such as smartphones and computers — where it can safeguard privacy and improve user experience — is highly desirable.

Now, a research team from Apple has created a generic system that enables “federated evaluation and tuning (FE&T)” systems on such end-user devices.

The objective of most current FL research is to learn a global neural network, while few FL studies have looked into the idea of model personalization in federated settings. The new Apple paper explores the application of FL systems for on-device personalization in an initial use case, automatic speech recognition (ASR). Ingesting data that is only available on-device, the system requires the evaluation and tuning of a personalization algorithm’s global parameters to create device-specific ASR language models.

The researchers studied how to improve user-specific ML system accuracy in the context of such ASR systems, especially when the personalization data remains inaccessible to the server-side.

The proposed system’s on-device components include a data store, task scheduler and results manager. The data store provides common on-device data retention policies to ensure that the amount of application-specific data stored is limited and recent. The task scheduler periodically downloads a list of available task descriptors when the system-level preconditions for device participation are met. The results manager sends back the task results and populates an on-device database that enables end-users to inspect data that is shared with the server, and also collects and sends health-related telemetry data to the server system.

The server components include a task manager, data manager and developer interface. The task manager stores and delivers all tasks and their attachments to the content delivery network, and can also retire tasks. The data manager is responsible for dropping any sensitive data from HTTP requests and forwarding results and telemetry information to a central database. The developer interface includes a web UI for monitoring task status and inspecting telemetry data and a Python script library.

To demonstrate federated tuning’s applicability to on-device personalization, the researchers described two specific large-scale personalization use cases: news personalization and ASR personalization.

The optimized parameters from the FT’s first run resulted in a 1.98 percent increase in daily article views, while FT’s second run saw a 1.87 percent increase in daily article views and a 0.90 percent increase in the daily time spent on the application.

The results show that for both user and generic cases, word error rates are lower after ASR personalization, validating the applicability of federated tuning to on-device personalization.

The paper Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications is on arXiv.

---

Author: Hecate He | Editor: Michael Sarazen

---

We know you don’t want to miss any news or research breakthroughs. Subscribe to our popular newsletter Synced Global AI Weekly to get weekly AI updates.