Secure Aggregation Scheme for Horizontal Fedetated Learning System

doi:10.3778/j.issn.1002-8331.2304-0287

Abstract

Abstract: A secure model aggregation scheme for privacy-preserving horizontal federated learning system is proposed in this paper. When homomorphic encryption is used in a horizontal federated learning system for privacy protection, the aggregation server can accurately detect Byzantine participants and realize the secure aggregation of the global model. The experimental results show that the proposed scheme can obtain a global model with high accuracy when there are Byzantine participants in the system, and will not bring too much computational and communication overhead to the federated learning system.

Key words: federated learning, secure aggregation, Byzantine attack, anomaly detection, privacy preserving

摘要： 针对隐私保护的横向联邦学习系统提出了一种模型安全聚合方案。在横向联邦学习系统利用同态加密进行隐私保护的情况下，服务器可以准确检测拜占庭节点发起的模型投毒攻击，避免异常本地模型参与全局模型的聚合。实验结果表明，所提方案可以保证在系统中存在拜占庭节点发动模型投毒攻击的情况下得到安全聚合的和高准确度的全局模型，并且不会为联邦学习系统带来过多的计算和通信开销。

关键词: 联邦学习, 安全聚合, 拜占庭攻击, 异常检测, 隐私保护

HUANG Xiuli, YU Pengfei, GAO Xianzhou. Secure Aggregation Scheme for Horizontal Fedetated Learning System[J]. Computer Engineering and Applications, 2024, 60(14): 294-305.

黄秀丽, 于鹏飞, 高先周. 横向联邦学习系统的安全聚合方法[J]. 计算机工程与应用, 2024, 60(14): 294-305.

References

[1] YANG Q, LIU Y, CHEN T, et al. Federated machine learning: concept and applications[J]. ACM Transactions on Intelligent Systems and Technology, 2019, 10(2): 1-19.
[2] LU Y, HUANG X, ZHANG K, et al. Blockchain empowered asynchronous federated learning for secure data sharing in internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2020, 69(4): 4298-4311.
[3] LU Y, HUANG X, DAI Y, et al. Differentially private asynchronous federated learning for mobile edge computing in urban informatics[J]. IEEE Transactions on Industrial Informatics, 2020, 16(3): 2134-2143.
[4] LU Y, HUANG X, DAI Y, et al. Blockchain and federated learning for privacy-preserved data sharing in industrial IoT[J]. IEEE Transactions on Industrial Informatics, 2020, 16(6): 4177-4186.
[5] PHONG L T, AONO Y, HAYASHI T, et al. Privacy-preserving deep learning via additively homomorphic encryption[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(5): 1333-1345.
[6] BONAWITZ K, IVANOV V, KREUTER B, et al. Practical secure aggregation for privacy-preserving machine learning[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017: 1175-1191.
[7] MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, 2017: 1273-1282.
[8] FUNG C, YOON C J M, BESCHASTNIKH I. Mitigating sybils in federated learning poisoning[J]. arXiv:1808.04866, 2018.
[9] CAO D, CHANG S, LIN Z, et al. Understanding distributed poisoning attack in federated learning[C]//Proceedings of the 25th IEEE International Conference on Parallel and Distributed Systems, 2019: 233-239.
[10] LI S, CHENG Y, LIU Y, et al. Abnormal client behavior detection in federated learning[J]. arXiv:1910.09933, 2019.
[11] YIN D, CHEN Y, KANNAN R, et al. Byzantine-robust distributed learning: towards optimal statistical rates[C]//Proceedings of the International Conference on Machine Learning, 2018: 5650-5659.
[12] RIVEST R L, ADLEMAN L M, DERTOUZOS M L. On data banks and privacy homomorphisms[J]. Foundations of Secure Computation, 1978, 4(11): 169-180.
[13] CANETTI R, FEIGE U, GOLDREICH O, et al. Adaptively secure multi-party computation[C]//Proceedings of the 28th Annual ACM Symposium on Theory of Computing, 1996: 639-648.
[14] GENTRY C. Fully homomorphic encryption using ideal lattices[C]//Proceedings of the 41st Annual ACM Symposium on Theory of Computing, 2009: 169-178.
[15] 崔建京, 龙军, 闵尔学, 等. 同态加密在加密机器学习中的应用研究综述[J]. 计算机科学, 2018, 45(4): 46-52.
CUI J J, LONG J, MIN E X, et al. Survey on application of homomorphic encryption in encrypted machine learning[J]. Computer Science, 2018, 45(4): 46-52.
[16] YAO A C. How to generate and exchange secrets[C]//Proceedings of the 27th Annual Symposium on Foundations of Computer Science, 1986: 162-167.
[17] DWORK C. Differential privacy: a survey of results[C]//Proceedings of the International Conference on Theory and Applications of Models of Computation, 2008: 1-19.
[18] REZA S, VITALY S. Privacy-preserving deep learning[C]//Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, 2015: 1310-1321.
[19] HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: information leakage from collaborative deep learning[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, 2017: 603-618.
[20] GEYER R C, KLEIN T, NABI M. Differentially private federated learning: a client level perspective[J]. arXiv:1712. 07557, 2017.
[21] XUAN G, ABHISHEK S, SRIKRISHNA K, et al. Preserving privacy in federated learning with ensemble cross-domain knowledge distillation[C]//Proceedings of the 36th AAAI Conference on Artificial Intelligence, 2022: 11891-11899.
[22] CHEN F, YUAN B, HAN Y J, et al. Efficient federated learning for privacy-preserving communication in IoT[J]. Computers & Security, 2021, 103: 102199.
[23] WANG J, GUO S, XIE X, et al. Protect privacy from gradient leakage attack in federated learning[C]//Proceedings of the IEEE INFOCOM 2022-IEEE Conference on Computer Communications, 2022: 580-589.
[24] CHEN W N, OZGUR A, KAIROUZ P. The Poisson binomial mechanism for unbiased federated learning with secure aggregation[C]//Proceedings of the International Conference on Machine Learning, 2022: 3490-3506.
[25] BRAILOVSKY V L. An approach to outlier detection based on bayesian probabilistic model[C]//Proceedings of the 13th International Conference on Pattern Recognition, 1996: 70-74.
[26] TAO Y, PI D. Unifying density-based clustering and outlier detection[C]//Proceedings of 2009 Second International Workshop on Knowledge Discovery and Data Mining, 2009: 644-647.
[27] 宋金玉, 郭一平, 王斌. DBSCAN聚类算法的参数配置方法研究[J]. 计算机技术与发展, 2019, 29(5): 44-48.
SONG J Y, GUO Y P, WANG B. Research on parameter configuration method of dbscan clustering algorithm[J]. Computer Technology and Development, 2019, 29(5): 44-48.
[28] SO J, AMIR S A. Byzantine-resilient secure federated learning[J]. IEEE Journal of Selected Areas in Communications, 2021, 39(7): 2168-2181.
[29] DONG Y, CHEN X, LI K, et al. FLOD: oblivious defender for private Byzantine-robust federated learning with dishonest-majority[C]//Proceeding of 26th European Symposium on Research in Computer Security, 2021: 497-518.