面向分层联邦学习框架的自适应异步聚合算法

doi:10.3778/j.issn.1002-8331.2412-0444

摘要/Abstract

摘要： 联邦学习作为一种分布式训练技术，在大规模神经网络模型训练得到广泛应用，但由于物联网设备之间的高通信成本、数据分布异质以及隐私安全等问题，对联邦学习训练带来巨大的挑战。针对上述问题，提出一种基于分层联邦学习的自适应异步聚合算法（HASFL）。客户端使用稀疏矩阵和本地差分隐私策略对本地模型进行处理并上传至边缘服务器，边缘服务器在进行边缘聚合时通过异常检测机制剔除恶意客户端设备。在边缘端通过动态分组和数据量加权的自适应异步聚合算法降低更新过程中的模型陈旧性差异对全局模型的影响。在服务端使用基于评分机制的自适应动态聚合算法对边缘服务器进行贡献评估动态调整其权重，以提高模型收敛速度和系统稳定性。实验结果表明，HASFL与FedAsync算法相比，达到目标精度的时间缩短约40%，并且在收敛速度和稳定性方面均优于其他对比算法。

关键词: 分层联邦学习, 差分隐私, 动态分组, 异步聚合, 异常检测, 自适应聚合

Abstract: Federated learning, as a distributed training technology, has been widely applied in the training of large-scale neural network models. However, due to issues such as high communication costs among Internet of things devices, heterogeneous data distribution, and privacy and security, it poses significant challenges to federated learning training. To address the above problems, an adaptive asynchronous aggregation algorithm based on hierarchical federated learning (HASFL) is proposed. Firstly, the client processes the local model using a sparse matrix and a local differential privacy policy and uploads it to the edge server. When the edge server performs edge aggregation, it eliminates malicious client devices through an anomaly detection mechanism. Secondly, at the edge end, the influence of model aging differences during the update process on the global model is reduced through an adaptive asynchronous aggregation algorithm based on dynamic grouping and data volume weighting. Finally, on the server side, an adaptive dynamic aggregation algorithm based on the scoring mechanism is used to evaluate the contributions of edge servers and dynamically adjust their weights to improve the convergence speed of the model and the stability of the system. The experimental results show that, compared with the FedAsync algorithm, HASFL shortens the time to achieve the target accuracy by approximately 40%, and is superior to other comparison algorithms in terms of convergence speed and stability.

Key words: hierarchical federated learning, differential privacy, dynamic grouping, asynchronous aggregation, anomaly detection, adaptive aggregation

徐飞, 申奥祥, 赵毅恒, 邓岩, 宁辛, 王星. 面向分层联邦学习框架的自适应异步聚合算法[J]. 计算机工程与应用, 2025, 61(19): 92-105.

XU Fei, SHEN Aoxiang, ZHAO Yiheng, DENG Yan, NING Xin, WANG Xing. Adaptive Asynchronous Aggregation Algorithm for Hierarchical Federated Learning Framework[J]. Computer Engineering and Applications, 2025, 61(19): 92-105.

参考文献

[1] TRAN A T, LUONG T D, PHAM X S. A novel privacy-preserving federated learning model based on secure multi-party computation[C]//Proceedings of the International Symposium on Integrated Uncertainty in Knowledge Modelling and Decision Making. Cham: Springer Nature Switzerland, 2023: 321-333.
[2] SHI Z S, YANG Z Y, HASSAN A, et al. A privacy preserving federated learning scheme using homomorphic encryption and secret sharing[J]. Telecommunication Systems, 2023, 82(3): 419-433.
[3] GE L N, LI H A, WANG X, et al. A review of secure federated learning: privacy leakage threats, protection technologies, challenges and future directions[J]. Neurocomputing, 2023, 561: 126897.
[4] ZHANG T, GAO L, HE C Y, et al. Federated learning for the Internet of things: applications, challenges, and opportunities[J]. IEEE Internet of Things Magazine, 2022, 5(1): 24-29.
[5] LI L, FAN Y X, TSE M, et al. A review of applications in federated learning[J]. Computers & Industrial Engineering, 2020, 149: 106854.
[6] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017: 1273-1282.
[7] ZHANG J C, LI S N. Blockchain-empowered vehicular intelligence: a perspective of asynchronous federated learning[J]. IEEE Internet of Things Magazine, 2024, 7(1): 74-80.
[8] WANG T, LIU Y, ZHENG X, et al. Edge-based communication optimization for distributed federated learning[J]. IEEE Transactions on Network Science and Engineering, 2022, 9(4): 2015-2024.
[9] WANG L L, ZHAO X Q, LU Z K, et al. Enhancing privacy preservation and trustworthiness for decentralized federated learning[J]. Information Sciences, 2023, 628: 449-468.
[10] LYU H Q, ZHANG Y X, WANG C, et al. Federated learning privacy incentives: reverse auctions and negotiations[J]. CAAI Transactions on Intelligence Technology, 2023, 8(4): 1538-1557.
[11] LI Y, YU H T, ZENG Y, et al. HFSA: a semi-asynchronous hierarchical federated recommendation system in smart city[J]. IEEE Internet of Things Journal, 2023, 10(21): 18808-18820.
[12] XIA Q, YE W, TAO Z Y, et al. A survey of federated learning for edge computing: research problems and solutions[J]. High-Confidence Computing, 2021, 1(1): 100008.
[13] SU Z, WANG Y T, LUAN T H, et al. Secure and efficient federated learning for smart grid with edge-cloud collaboration[J]. IEEE Transactions on Industrial Informatics, 2022, 18(2): 1333-1344.
[14] LIU L M, ZHANG J, SONG S H, et al. Client-edge-cloud hierarchical federated learning[C]//Proceedings of the 2020 IEEE International Conference on Communications. Piscataway: IEEE, 2020: 1-6.
[15] CHENG K W, FAN T, JIN Y L, et al. SecureBoost: a lossless federated learning framework[J]. IEEE Intelligent Systems, 2021, 36(6): 87-98.
[16] LI T, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[C]//Proceedings of the 3rd Conference on Machine Learning and Systems, 2020: 429-450.
[17] XU C H, QU Y Y, XIANG Y, et al. Asynchronous federated learning on heterogeneous devices: a survey[J]. Computer Science Review, 2023, 50: 100595.
[18] SO J, GüLER B, AVESTIMEHR A S. Turbo-aggregate: breaking the quadratic aggregation barrier in secure federated learning[J]. IEEE Journal on Selected Areas in Information Theory, 2021, 2(1): 479-489.
[19] XIE C, KOYEJO S, GUPTA I. ?Asynchronous federated optimization[J]. arXiv:1903.03934, 2019.
[20] LIU J C, XU H L, WANG L, et al. Adaptive asynchronous federated learning in resource-constrained edge computing[J]. IEEE Transactions on Mobile Computing, 2023, 22(2): 674-690.
[21] CHEN Z Y, LIAO W X, HUA K, et al. Towards asynchronous federated learning for heterogeneous edge-powered Internet of things[J]. Digital Communications and Networks, 2021, 7(3): 317-326.
[22] HAO J S, ZHAO Y C, ZHANG J L. Time efficient federated learning with semi-asynchronous communication[C]//Proceedings of the 2020 IEEE 26th International Conference on Parallel and Distributed Systems. Piscataway: IEEE, 2020: 156-163.
[23] HU C H, CHEN Z, LARSSON E G. Scheduling and aggregation design for asynchronous federated learning over wireless networks[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(4): 874-886.
[24] LI Z X, HUANG C P, GAI K K, et al. Asyfed: accelerated federated learning with asynchronous communication mechanism[J]. IEEE Internet of Things Journal, 2023, 10(10): 8670-8683.
[25] YU M R, CHOI J, LEE J, et al. Staleness aware semi-asynchronous federated learning[J]. Journal of Parallel and Distributed Computing, 2024, 193: 104950.
[26] CAO Y, LIU D T, ZHANG S Q, et al. T-FedHA: a trusted hierarchical asynchronous federated learning framework for Internet of Things[J]. Expert Systems with Applications, 2024, 245: 123006.
[27] DENG Y H, LYU F, XIA T X, et al. A communication-efficient hierarchical federated learning framework via shaping data distribution at edge[J]. IEEE/ACM Transactions on Networking, 2024, 32(3): 2600-2615.
[28] LI H, ZHOU S, YUAN B, et al. Optimizing intelligent edge computing resource scheduling based on federated learning[J]. Journal of Knowledge Learning and Science Technology, 2024, 3(3): 235-260.
[29] LIANG F Y, YANG Q L, LIU R Q, et al. Semi-synchronous federated learning protocol with dynamic aggregation in Internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2022, 71(5): 4677-4691.
[30] LI Q B, DIAO Y Q, CHEN Q, et al. Federated learning on non-IID data silos: an experimental study[C]//Proceedings of the 2022 IEEE 38th International Conference on Data Engineering. Piscataway: IEEE, 2022: 965-978.