FEDn is an open-source framework for federated machine learning, developed in the Computational Science and Engineering group at Uppsala University together with Scaleout Systems. I worked on it as a postdoctoral researcher from 2020 to 2022, on the framework itself and on applications built with it.
The problem
By 2020, several open-source projects made it easy to experiment with federated learning: to try new ways of combining model updates, or new privacy techniques, inside a single machine. Far fewer addressed the distributed-computing side of the problem: scalability and resilience when real clients, spread over many sites and networks, take part. For a data scientist, going from an experimental sandbox to a real, geographically distributed deployment was a big step.
The approach
FEDn was designed as a production-grade, hierarchical framework that makes this step easy. The same code runs in a pseudo-distributed mode on a laptop and in a horizontally scalable deployment across many sites.
- Three tiers. Clients train on their own data. Combiners aggregate the updates of the clients connected to them, and a reducer combines their results into the global model, under the control of a controller. More combiners can be added as the number of clients grows.
- Framework-agnostic. Clients can use any machine learning library, such as PyTorch, TensorFlow/Keras or scikit-learn.
- Both settings. It supports cross-silo use, with a few organisations, and cross-device use, with many small devices.
In the CCGrid 2022 paper, we presented the architecture and implementation and showed its scalability and efficiency in two case studies, one cross-silo and one cross-device.
Results and applications
- FedQAS is a privacy-preserving machine reading system: transformer models for question answering, trained on private text at several organisations without pooling it. It was built with FEDn and evaluated on the SQuAD dataset.
- FedBot is a proof-of-concept chatbot for customer support that combines transformer models with federated learning, so that customer conversations stay with the company that holds them.
- Efficient training at the edge. Training only a random subset of a model’s layers on each device cut data transfer by around 75% when a quarter of the layers were trained, and by 53% at half, without hurting the global model’s accuracy. The experiments were run with FEDn.
Today
FEDn has since become the engine of Scaleout Edge, Scaleout Systems’ commercial platform. The last open-source release of the fedn package is on PyPI. My blog post on federated learning frameworks compares FEDn with other frameworks.