Federated Noisy Client Learning

Federated Noisy Client Learning

IEEE Transactions on Neural Networks and Learning Systems

10.1109/TNNLS.2023.3336050

http://dx.doi.org/10.1109/tnnls.2023.3336050

Kahou Tam, Li Li, Bo Han, Chengzhong Xu, Huazhu Fu

01 December 2023

Tam, K., Li, L., Han, B., Xu, C., & Fu, H. (2024). Federated Noisy Client Learning. IEEE Transactions on Neural Networks and Learning Systems, 1–14. https://doi.org/10.1109/tnnls.2023.3336050

Federated learning (FL) collaboratively trains a shared global model depending on multiple local clients, while keeping the training data decentralized to preserve data privacy. However, standard FL methods ignore the noisy client issue, which may harm the overall performance of the shared model. We first investigate the critical issue caused by noisy clients in FL and quantify the negative impact of the noisy clients in terms of the representations learned by different layers. We have the following two key observations: 1) the noisy clients can severely impact the convergence and performance of the global model in FL and 2) the noisy clients can induce greater bias in the deeper layers than the former layers of the global model. Based on the above observations, we propose federated noisy client learning (Fed-NCL), a framework that conducts robust FL with noisy clients. Specifically, Fed-NCL first identifies the noisy clients through well estimating the data quality and model divergence. Then robust layerwise aggregation is proposed to adaptively aggregate the local models of each client to deal with the data heterogeneity caused by the noisy clients. We further perform label correction on the noisy clients to improve the generalization of the global model. Experimental results on various datasets demonstrate that our algorithm boosts the performances of different state-of-the-art systems with noisy clients. Our code is available at https://github.com/TKH666/Fed-NCL.

Publisher Copyright

This research / project is supported by the A*STAR - Career Development Fund
Grant Reference no. : C222812010

© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

https://oar.a-star.edu.sg/communities-collections/articles/20721

2162-2388
2162-237X

Institute of High Performance Computing