Federated Learning From Theory To Practice To Advanced Topics

Short Description

ML often centralizes data for training, weakening data control and raising privacy, security, efficiency concerns—especially on edge devices. Federated Learning (FL) trains models without moving raw data, improving privacy and reducing transfer costs. This course moves from theory to practice, mixing fundamentals with advanced topics (personalization, hierarchy, privacy, robustness) and hands-on labs. We cover core FL variants (cross-device, cross-silo, hierarchical, personalized), the role of data heterogeneity (IID vs. non-IID), and its impact on convergence, fairness, and robustness. Flower is used to convert a centralized training loop into a federated one with minimal changes, and FedArtML to generate controllable non-IID datasets for rigorous evaluation. We also examine threats—poisoning, backdoors, Byzantine behavior, membership inference—and practical defenses (robust aggregation, sanitization, differential privacy, secure aggregation). Special attention goes to Internet of Things (IoT) scenarios, where devices with limited processing, storage, and energy join training using resource-aware strategies. Lectures, demos, and labs guide participants to implement an end-to-end FL pipeline, from data generation to attack mitigation.

Additional information can be found at official website of the course.

Course Objectives

  • Understand the FL computation model and its motivations (privacy, regulation, efficiency).
  • Distinguish and apply variants: cross-device, cross-silo, hierarchical, and personalized FL.
  • Master IID vs. non-IID notions and quantify their effect on performance and stability.
  • Use Flower to transform a centralized (PyTorch/TensorFlow) training routine into a federated one.
  • Generate and control heterogeneity with FedArtML for reproducible experimentation.
  • Implement and compare optimization/aggregation algorithms (FedAvg, FedProx, FedOpt) under diverse scenarios.
  • Recognize relevant cyber-attacks (data/model poisoning, backdoors, Byzantine behavior, inference) and apply practical defenses (Krum/median/trimmed-mean, sanitization, DP, secure aggregation).
  • Design a mini-project: an FL pipeline under non-IID data with evaluation of a poisoning attack and mitigation.

The course builds on Andrea Vitaletti’s 2025 ECI course via its treatment of Differential Privacy and consensus/Byzantine-robust aggregation mechanisms.

Course Plan & Material

  • Lecture 1: Monday, July 27, 2026. Foundations of Federated Learning
The federated learning computation model: why training moves to the data instead of the data moving to training, and what this buys in terms of privacy, regulatory compliance (e.g., GDPR-style data-locality constraints), and communication efficiency. System architecture of a federated deployment (server, clients, communication rounds) and the main FL variants — cross-device, cross-silo, hierarchical, personalized. Introduction to differential privacy (DP): noise mechanisms and gradient clipping as a way to bound how much any single client's data can influence, or be recovered from, the exchanged model weights; a brief look at secure aggregation as a complementary, cryptographic route to the same goal.
Lab: Environment setup (Python, PyTorch/TensorFlow, Flower). Review of a centralized training loop as the starting point for the rest of the course. Implement a simple DP mechanism (clipping + Gaussian noise) directly on gradients/weights and observe the privacy–utility trade-off as the noise scale increases.
Material:
  • Lecture 2: Tuesday, July 28, 2026. Data Heterogeneity: IID vs. Non-IID
Why the IID assumption used in Lecture 1 rarely holds in practice: clients generate their own local data, so quantity, labels, and features naturally differ from client to client. We formalize this as quantity skew, label skew, and feature skew, and introduce three ways to quantify how non-IID a federation is by comparing per-client label distributions to the global one: Hellinger Distance, Jensen–Shannon Distance, and Earth Mover's (Wasserstein) Distance. We also preview why heterogeneity matters — it is the main driver of slow or unstable convergence, of the need for personalized and hierarchical FL architectures, and of the fairness and robustness issues covered in Lectures 4 and 5.
Lab: Use FedArtML to synthesize non-IID client partitions with tunable heterogeneity (Dirichlet and similarity-based partitioning). Compute Hellinger, Jensen–Shannon, and Earth Mover's Distance per client and in aggregate, and relate the resulting values to a simple non-IID-ness scale. Discuss, with a short personalization baseline, how increasing heterogeneity degrades a single shared global model.
Material:
  • Lecture Slides in PDF
  • FedArtML - a python-based software library providing metrics for quantifying the degree of non-IID (non-IID-ness) data residing across entities participating in decentralized data and methods to create synthetic datasets suitable for federated learning.
  • Lecture 3: Wednesday, July 29, 2026. Personalized FL and Client Selection strategies
Three federated optimization/aggregation algorithms and the specific problem each was designed to fix: FedAvg's plain local-SGD-then-average step, FedProx's proximal term for tolerating heterogeneous local computation and client drift, and FedOpt's use of adaptive server-side optimizers (e.g., Adam, Adagrad, Yogi) in place of simple averaging. We define the evaluation criteria used to compare them under non-IID data: performance disparity across clients (variance/IQR of accuracy and worst-client accuracy), convergence time to a chosen accuracy threshold, and communication volume per round (bytes of updates and metadata exchanged).
Lab: Migrate the centralized model from Lecture 1 to Flower; configure the aggregation strategy and training loop. Train on the Day-2 non-IID partitions while logging disparity, convergence time, and per-round communication volume, then compare FedAvg, FedProx, and FedOpt runs against one another on these three criteria.
Material:
  • Lecture 4: Thursday, July 30, 2026. Fairness in Federated Learning
How a federated model can score well in aggregate while systematically disadvantaging a protected group or a subset of clients, and why this risk is amplified by the non-IID conditions studied in Lecture 2. Notions of fairness relevant to FL: demographic parity difference, equalized-odds/TPR–FPR gaps between groups, and worst-client fairness as a client-centric alternative to group-based metrics. Mitigation strategies: client/example re-weighting, fairness-aware training objectives and constraints, and personalization targeted at protected groups or underperforming clients.
Lab: Practical exercises on a dataset with protected attributes (e.g., gender/age). Implement demographic parity, equalized-odds, and worst-client fairness metrics on a trained global model; apply at least one mitigation strategy within the FL pipeline; analyze the resulting fairness–accuracy trade-off.
Material:
  • Lecture 5: Friday, July 31, 2026. Security, Privacy, and Robustness in Federated Learning
The federated threat model: a malicious client or coalition of clients can submit corrupted updates (data or model poisoning), plant a backdoor that survives across rounds, or behave arbitrarily (Byzantine behavior); a curious server or eavesdropper can attempt membership inference or gradient-based reconstruction against individual clients. We define adversarial robustness, accuracy under attack (absolute/relative drop versus a clean baseline), and fault tolerance (degradation as a function of the malicious/failed client fraction, worst case, and recovery time), and connect these to the defenses that address them: robust aggregation (Krum, coordinate-wise median, trimmed mean), input/update sanitization, differential privacy, and secure aggregation.
Lab: Simulate a poisoning and a backdoor attack in the Flower pipeline built over the course; deploy Krum, median, and trimmed-mean aggregation as defenses and compare them against plain FedAvg. Measure and report adversarial robustness, accuracy-under-attack, and fault tolerance, and discuss the resulting trade-offs against the communication and fairness metrics from Lectures 3 and 4.
Material:

Individual Project

Relevant Bibliography

Textbooks

  • Jeno, George. Federated Learning with Python: Design and implement a federated learning system and develop applications using existing frameworks. Packt Publishing Ltd, 2022.
  • Ludwig, Heiko, and Nathalie Baracaldo, eds. Federated learning: A comprehensive overview of methods and applications. Cham: Springer, 2022.

Foundational algorithms (Lectures 1 & 3)

  • McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; y Arcas, B. A. Communication-Efficient Learning of Deep Networks from Decentralized Data. Artificial Intelligence and Statistics (AISTATS), PMLR, 2017, pp. 1273–1282.
  • Li, T.; Sahu, A. K.; Zaheer, M.; Sanjabi, M.; Talwalkar, A.; Smith, V. Federated Optimization in Heterogeneous Networks. Proceedings of Machine Learning and Systems (MLSys), 2020.
  • Reddi, S.; Charles, Z.; Zaheer, M.; Garrett, Z.; Rush, K.; Konečný, J.; Kumar, S.; McMahan, H. B. Adaptive Federated Optimization. International Conference on Learning Representations (ICLR), 2021.
  • Liu, L.; Zhang, J.; Song, S.; Letaief, K. B. Client-Edge-Cloud Hierarchical Federated Learning. IEEE International Conference on Communications (ICC), 2020.
  • Fallah, A.; Mokhtari, A.; Ozdaglar, A. Personalized Federated Learning: A Meta-Learning Approach. Advances in Neural Information Processing Systems (NeurIPS), 2020.
  • Nishio, T.; Yonetani, R. Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge. IEEE International Conference on Communications (ICC), 2019.

Data heterogeneity and non-IID quantification (Lecture 2)

  • Jimenez G., D. M.; Solans, D.; Heikkila, M.; Vitaletti, A.; Kourtellis, N.; Anagnostopoulos, A.; Chatzigiannakis, I. Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions, arXiv:2411.12377, 2024. doi: 10.48550/arXiv.2411.12377.
  • Gutierrez, D. M. J.; Anagnostopoulos, A.; Chatzigiannakis, I.; Vitaletti, A. FedArtML: A Tool to Facilitate the Generation of Non-IID Datasets in a Controlled Way to Support Federated Learning Research, IEEE Access, vol. 12, pp. 81004–81016, 2024. doi: 10.1109/ACCESS.2024.3410026.
  • Jimenez-Gutierrez, D. M.; Hassanzadeh, M.; Anagnostopoulos, A.; Chatzigiannakis, I.; Vitaletti, A. A Thorough Assessment of the Non-IID Data Impact in Federated Learning, Journal of Industrial Information Integration, vol. 50, 2026, 101052. doi: 10.1016/j.jii.2025.101052.

Personalization and client selection under heterogeneity (Lectures 2 & 3)

  • Jimenez-Gutierrez, D. M.; Hassanzadeh, M.; Anagnostopoulos, A.; Chatzigiannakis, I.; Vitaletti, A. Clust-PSI-PFL: A Population Stability Index Approach for Clustered Non-IID Personalized Federated Learning, 40th IEEE International Parallel & Distributed Processing Symposium (IPDPS), 2026. Preprint: arXiv:2512.20363.
  • PSI-PFL: Population Stability Index for Client Selection in non-IID Personalized Federated Learning. IPDPS 2026.
  • PFAE: Personalized Federated Learning for Anomaly Detection Over Heterogeneous IoT Domains, INFOCOM 2026.

Fairness in Federated Learning (Lecture 4)

  • Hardt, M.; Price, E.; Srebro, N. Equality of Opportunity in Supervised Learning. Advances in Neural Information Processing Systems (NeurIPS), 2016.
  • Li, T.; Sanjabi, M.; Beirami, A.; Smith, V. Fair Resource Allocation in Federated Learning. International Conference on Learning Representations (ICLR), 2020.
  • Mocerino, P.; Jimenez-Gutierrez, D. M.; Chatzigiannakis, I. FairChoice: Balancing Performance and Group Fairness in Federated Learning with Non-IID Data, 2025 IEEE International Smart Cities Conference (ISC2), Patras, Greece, 2025, pp. 1–6. doi: 10.1109/ISC266238.2025.11293299.

Security, privacy, and robustness (Lecture 5)

  • Blanchard, P.; El Mhamdi, E. M.; Guerraoui, R.; Stainer, J. Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent. Advances in Neural Information Processing Systems (NeurIPS), 2017.
  • Yin, D.; Chen, Y.; Kannan, R.; Bartlett, P. Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates. International Conference on Machine Learning (ICML), 2018.
  • Bagdasaryan, E.; Veit, A.; Hua, Y.; Estrin, D.; Shmatikov, V. How to Backdoor Federated Learning. Artificial Intelligence and Statistics (AISTATS), 2020.
  • Shokri, R.; Stronati, M.; Song, C.; Shmatikov, V. Membership Inference Attacks Against Machine Learning Models. IEEE Symposium on Security and Privacy (S&P), 2017.
  • Bonawitz, K.; Ivanov, V.; Kreuter, B.; Marcedone, A.; McMahan, H. B.; Patel, S.; Ramage, D.; Segal, A.; Seth, K. Practical Secure Aggregation for Privacy-Preserving Machine Learning. ACM Conference on Computer and Communications Security (CCS), 2017.
  • McMahan, H. B.; Ramage, D.; Talwar, K.; Zhang, L. Learning Differentially Private Recurrent Language Models. International Conference on Learning Representations (ICLR), 2018.
  • Jimenez-Gutierrez, D. M.; Falkouskaya, Y.; Hernandez-Ramos, J. L.; Anagnostopoulos, A.; Chatzigiannakis, I.; Vitaletti, A. On the Security and Privacy of Federated Learning: A Survey with Attacks, Defenses, Frameworks, Applications, and Future Directions, arXiv:2508.13730, 2025. doi: 10.48550/arXiv.2508.13730.