Review of Research on Information Security in Federated Learning

doi:10.3778/j.issn.1002-8331.2303-0332

Abstract

Abstract: As a new machine learning technology, federated learning allows participants to complete collaborative training and obtain global models through parameter interaction without sharing original data. It provides a new paradigm for breaking data silos and integrating data resources and has become a research hotspot in the field of artificial intelligence. However, federated learning still faces many security risks. This paper systematically analyzes and classifies the latest research results in the field of federated learning at home and abroad. Taking the training process of the federated learning model as a clue, it analyzes the security threats that may exist in the system during each process, studies the mechanism and characteristics of different security threats, and classifies them according to the degree of threat. Based on this, the paper studies the current advanced defense strategies. Finally, it discusses the main challenges and future development directions of federated learning in order to promote the safe landing and promotion of federated learning applications.

Key words: federated learning, data security, system threat, defense strategy

摘要： 联邦学习作为一种新兴的机器学习技术，允许多方在不共享原始数据的前提下通过参数交互完成协同训练，生成全局模型，为打破数据孤岛、整合数据资源提供了新范式，成为人工智能领域的一大研究热点。但联邦学习依然面临诸多安全风险。对联邦学习领域的国内外最新研究成果进行系统分析和分类，以联邦学习模型训练过程为线索，分析每个过程中系统可能存在的安全威胁，研究不同安全威胁的机理和特点，并按照威胁程度对其进行分类，在此基础上，研究当前先进的防御策略；最后，探讨了联邦学习面临的主要挑战和未来发展方向，旨在推动联邦学习应用安全落地和推广。

关键词: 联邦学习, 数据安全, 系统威胁, 防御策略

DUAN Xinru, CHEN Guirong, CHEN Aiwang, CHEN Chen, JI Weifeng. Review of Research on Information Security in Federated Learning[J]. Computer Engineering and Applications, 2024, 60(3): 61-77.

段昕汝, 陈桂茸, 陈爱网, 陈晨, 姬伟峰. 联邦学习中的信息安全问题研究综述[J]. 计算机工程与应用, 2024, 60(3): 61-77.

References

[1] MCMAHAN H B, MOORE E, RAMAGE D, et al. Federated learning of deep networks using model averaging[J]. arXiv:1602.05629, 2016.
[2] MCMAHAN B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Lauderdale, Apr 20-22, 2017: 1273-1282.
[3] 杨强. AI与数据隐私保护: 联邦学习的破解之道[J].信息安全研究, 2019, 5(11): 961-965.
YANG Q. AI and data privacy protection: the way to federated learning[J]. Journal of Information Security Research, 2019, 5(11): 961-965.
[4] WU X, ZHENG H, DOU Z, et al. A novel privacy-preserving federated genome-wide association study framework and its application in identifying potential risk variants in ankylosing spondylitis[J]. Briefings in Bioinformatics, 2021, 22(3): 90.
[5] CHE S, KONG Z, PENG H, et al. Federated multi-view learning for private medical data integration and analysis[J]. ACM Transactions on Intelligent Systems and Technology (TIST), 2022, 13(4): 1-23.
[6] 王生生, 陈境宇, 卢奕南.基于联邦学习和区块链的新冠肺炎胸部CT图像分割[J].吉林大学学报(工学版), 2021, 51(6): 2164-2173.
WANG S S, CHEN J Y, LU Y N. COVID-19 chest CT image segmentation based on federated learning and blockchain[J]. Journal of Jilin University(Engineering and Technology Edition), 2021, 51(6): 2164-2173.
[7] HARD A, RAO K, MATHEWS R, et al. Federated learning for mobile keyboard prediction[J]. arXiv:1811.03604, 2018.
[8] 王蓉, 马春光, 武朋.基于联邦学习和卷积神经网络的入侵检测方法[J].信息网络安全, 2020(4): 47-54.
WANG R, MA C G, WU P. An intrusion detection method based on federated learning and convolutional neural network[J]. Netinfo Security, 2020(4): 47-54.
[9] 任涛, 金若辰, 罗咏梅.融合区块链与联邦学习的网络入侵检测算法[J].信息网络安全, 2021(7): 27-34.
REN T, JIN R C, LUO Y M. Network intrusion detection algorithm integrating blockchain and federated learning[J]. Netinfo Security, 2021(7): 27-34.
[10] 白宏鹏, 邓东旭, 许光全, 等.基于联邦学习的入侵检测机制研究[J].信息网络安全, 2022(1): 46-54.
BAI H P, DENG D X, XU G Q, et al. Research on intrusion detection mechanism based on federated learning[J]. Netinfo Security, 2022(1): 46-54.
[11] 杨强, 童咏昕, 王晏晟, 等.群体智能中的联邦学习算法综述[J]. 智能科学与技术学报, 2022, 4(1): 29-44.
YANG Q, TONG Y X, WANG Y S, et al. A survey on federated learning in crowd intelligence[J]. Chinese Journal of Intelligent Science and Technology, 2022, 4(1): 29-44.
[12] 杨强.联邦学习: 人工智能的最后一公里[J].智能系统学报, 2020, 15(1): 183-186.
YANG Q. Federated learning: the last on kilometer of artificial intelligence[J]. CAAI Transactions on Intelligent Systems, 2020, 15(1): 183-186.
[13] YANG Q, LIU Y, CHEN T, et al. Federated machine learning: concept and applications[J]. ACM Transactions on Intelligent Systems and Technology (TIST), 2019, 10(2): 1-19.
[14] LI X, HUANG K, YANG W, et al. On the convergence of fedavg on non-IID data[J]. arXiv:1907.02189, 2019.
[15] KARIMIREDDY S P, KALE S, MOHRI M, et al. SCAFFOLD: stochastic controlled averaging for on-device federated learning[J]. arXiv:1910.06378, 2019.
[16] LI T, SAHU A K, ZAHEER M, et al. Federated optimization in heterogeneous networks[J]. Proceedings of Machine Learning and Systems, 2020, 2: 429-450.
[17] T DINH C, TRAN N, NGUYEN J. Personalized federated learning with moreau envelopes[C]//Advances in Neural Information Processing Systems, 2020: 21394-21405.
[18] BONAWITZ K, EICHNER H, GRIESKAMP W, et al. Towards federated learning at scale: system design[J]. Proceedings of Machine Learning and Systems, 2019, 1: 374-388.
[19] BONAWITZ K, IVANOV V, KREUTER, B, et al. Practical secure aggregation for privacy-preserving machine learning[C]//Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, Oct 30-Nov 3, 2017: 1175-1191.
[20] ZHANG X, GU H, FAN L, et al. No free lunch theorem for security and utility in federated learning[J]. ACM Transactions on Intelligent Systems and Technology, 2022, 14(1): 1-35.
[21] BHOWMICK A, DUCHI J, FREUDIGER J, et al. Protection against reconstruction and its applications in private federated learning[J]. arXiv:1812.00984, 2018.
[22] BOUACIDA N, MOHAPATRA P. Vulnerabilities in federated learning[J]. IEEE Access, 2021, 9: 63229-63249.
[23] ABDULRAHMAN S, TOUT H, OULD-SLIMANE H, et al. A survey on federated learning: the journey from centralized to distributed on-site learning and beyond[J]. IEEE Internet of Things Journal, 2020, 8(7): 5476-5497.
[24] WANG Z, SONG M, ZHANG Z, et al. Beyond inferring class representatives: user-level privacy leakage from federated learning[C]//Proceedings of the IEEE INFOCOM 2019-IEEE Conference on Computer Communications, Paris, Apr 29-May 2, 2019: 2512-2520.
[25] MAMMEN P M. Federated learning: opportunities and challenges[J]. arXiv:2101.05428, 2021.
[26] MELIS L, SONG C, DE CRISTOFARO E, et al. Exploiting unintended feature leakage in collaborative learning[C]//Proceedings of the 2019 IEEE Symposium on Security and privacy (SP), San Francisco, May 19-23, 2019: 691-706.
[27] ZHU L, LIU Z, HAN S. Deep leakage from gradients[C]//Advances in Neural Information Processing Systems, 2019.
[28] NIKNAM S, DHILLON H S, REED J H. Federated learning for wireless communications: Motivation, opportunities, and challenges[J]. IEEE Communications Magazine, 2020, 58(6): 46-51.
[29] MA C, LI J, DING M, et al. On safeguarding privacy and security in the framework of federated learning[J]. IEEE Network, 2020, 34(4): 242-248.
[30] CARLINI N, LIU C, KOS J, et al. The secret sharer: measuring unintended neural network memorization & extracting secrets[J]. arXiv:1802.08232, 2018.
[31] GEIPING J, BAUERMEISTER H, DR?GE H, et al. Inverting gradients-how easy is it to break privacy in federated learning? [C]//Advances in Neural Information Processing Systems, 2020: 16937-16947.
[32] JAGIELSKI M, OPREA A, BIGGIO B, et al. Manipulating machine learning: poisoning attacks and countermeasures for regression learning[C]//Proceedings of the IEEE Symposium on Security and Privacy (SP), San Francisco, May 21-23, 2018: 19-35.
[33] SHAFAHI A, HUANG W R, NAJIBI M, et al. Poison frogs! targeted clean-label poisoning attacks on neural networks[C]//Advances in Neural Information Processing Systems, 2018.
[34] 张义莲, 颜晟, 朱旻捷, 等.机器学习系统毒化攻击综述[J]. 通信技术, 2020, 53(3): 535-542.
ZHANG Y L, YAN C, ZHU M J, et al.Overview on poisoning attacks against machine learning system[J]. Communications Technology, 2020, 53(3): 535-542.
[35] SUN G, CONG Y, DONG J, et al. Data poisoning attacks on federated machine learning[J]. IEEE Internet of Things Journal, 2021, 9(13): 11365-11375.
[36] TOLPEGIN V, TRUEX S, GURSOY M E, et al. Data poisoning attacks against federated learning systems[C]//Proceedings of the 25th European Symposium on Research in Computer Security, Guildford, Sep 14-18, 2020: 480-501.
[37] DOKU R, RAWAT D B. Mitigating data poisoning attacks on a federated learning-edge computing network[C]//Proceedings of the 18th Annual Consumer Communications & Networking Conference (CCNC), Las Vegas, Jan 9-12, 2021: 1-6.
[38] BHAGOJI A N, CHAKRABORTY S, MITTAL P, et al. Analyzing federated learning through an adversarial lens[C]//Proceedings of the 36th International Conference on Machine Learning, Long Beach, Jun 9-15, 2019: 634-643.
[39] FANG M, CAO X, JIA J, et al. Local model poisoning attacks to {Byzantine-Robust} federated learning[C]//Proceedings of the 29th USENIX Security Symposium, Boston, Aug 12-14, 2020: 1605-1622.
[40] SUN Z, KAIROUZ P, SURESH A T, et al. Can you really backdoor federated learning?[J]. arXiv:1911.07963, 2019.
[41] BAGDASARYAN E, VEIT A, HUA Y, et al. How to backdoor federated learning[C]//Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, Aug 26-28, 2020: 2938-2948.
[42] XIE C, HUANG K, CHEN P Y, et al. DBA: distributed backdoor attacks against federated learning[C]//Proceedings of the International Conference on Learning Representations, Apr 26-May 1, 2020.
[43] WANG H, SREENIVASAN K, RAJPUT S, et al. Attack of the tails: yes, you really can backdoor federated learning[C]//Advances in Neural Information Processing Systems, 2020: 16070-16084.
[44] BLANCHARD P, EL MHAMDI E M, GUERRAOUI R, et al. Machine learning with adversaries: byzantine tolerant gradient descent[C]//Advances in Neural Information Processing Systems, 2017.
[45] LAMPORT L, SHOSTAK R, PEASE M. The Byzantine generals problem[J] ACM Transactions on Programming Languages and Systems, 2019, 4(3) 382-401.
[46] GOUISSEM A, ABUALSAUD K, YAACOUB E, et al. Federated learning stability under Byzantine attacks[C]//Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC), Austin, Apr 10-13, 2022. Piscataway: IEEE, 2022: 572-577.
[47] GHOSH A, HONG J, YIN D, et al. Robust federated learning in a heterogeneous environment[J]. arXiv:1906.06629, 2019.
[48] SHI J, WAN W, HU S, et al. Challenges and approaches for mitigating byzantine attacks in federated learning[C]//Proceedings of the IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Wuhan, Dec 9-11, 2022. Piscataway: IEEE, 2022: 139-146.
[49] ZHAO P, CAO Z, JIANG J, et al. Practical private aggregation in federated learning against inference attack[J]. IEEE Internet of Things Journal, 2022, 10(1): 318-329.
[50] LAM M, WEI G Y, BROOKS D, et al. Gradient disaggregation: breaking privacy in federated learning by reconstructing the user participant matrix[C]//Proceedings of the 38th International Conference on Machine Learning, Jul 18-24, 2021: 5959-5968.
[51] YIN H, MALLYA A, VAHDAT A, et al. See through gradients: Image batch recovery via gradinversion[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 19-25, 2021. Piscataway: IEEE, 2021: 16337-16346.
[52] JIN X, CHEN P Y, HSU C Y, et al. CAFE: catastrophic data leakage in vertical federated learning[C]//Advances in Neural Information Processing Systems, 2021: 994-1006.
[53] LEBRUN T, BOUTET A, AALMOES J, et al. MixNN: protection of federated learning against inference attacks by mixing neural network layers[C]//Proceedings of the 23rd ACM/IFIP International Middleware Conference, Quebec Nov 7-11, 2022. New York: ACM, 2022: 135-147.
[54] LYU L, CHEN C. A novel attribute reconstruction attack in federated learning[J]. arXiv:2108.06910, 2021.
[55] HU H, SALCIC Z, SUN L, et al. Source inference attacks in federated learning[[C]//Proceedings of the IEEE International Conference on Data Mining (ICDM), Auckland, Dec 7-10, 2021. Piscataway: IEEE, 2021: 1102-1107.
[56] GU Y, BAI Y, XU S. CS-MIA: membership inference attack based on prediction confidence series in federated learning[J]. Journal of Information Security and Applications, 2022, 67: 103201.
[57] VO H, TANG M, ZHENG X, et al. BI-GAN: batch inversion membership inference attack on federated learning[C]//Proceedings of the 17th ACM Workshop on Mobility in the Evolving Internet Architecture, Sydney, Oct 21, 2022. New York: ACM, 2022: 31-36.
[58] LUO X, WU Y, XIAO X, et al. Feature inference attack on model predictions in vertical federated learning[C]//Proceedings of the IEEE 37th International Conference on Data Engineering (ICDE), Chania, Apr 19-22, 2021. Piscataway: IEEE, 2021: 181-192.
[59] 李少波, 杨磊, 李传江, 等.联邦学习概述: 技术、应用及未来[J].计算机集成制造系统, 2022, 28(7): 2119-2138.
LI S B, YANG L, LI C J, et al. Overview of federated learning: technology, applications and future[J]. Computer Integrated Manufacturing Systems, 2022, 28(7): 2119-2138.
[60] BELL J H, BONAWITZ K A, GASCóN A, et al. Secure single-server aggregation with (poly) logarithmic overhead[C]//Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, Nov 9-13, 2020. New York: ACM, 2020: 1253-1269.
[61] ANDREINA S, MARSON G A, M?LLERING H, et al. Baffle: backdoor detection via feedback-based federated learning[C]//Proceedings of the IEEE 41st International Conference on Distributed Computing Systems (ICDCS), DC, Jul 7-10, 2021. Piscataway: IEEE, 2021: 852-863.
[62] XIE C, CHEN M, CHEN P Y, et al. CRFL: certifiably robust federated learning against backdoor attacks[C]//Proceedings of the 38th International Conference on Machine Learning, Jul 18-24, 2021: 11372-11382.
[63] SUN J, LI A, DIVALENTIN L, et al. Fl-wbc: enhancing robustness against model poisoning attacks in federated learning from a client perspective[C]//Advances in Neural Information Processing Systems, 2021: 12613-12624.
[64] LYCKLAMA H, BURKHALTER L, VIAND A, et al. RoFL: robustness of secure federated learning[C]//Proceedings of the IEEE Symposium on Security and Privacy (SP), San Francisco, May 22-26, 2023: 453-476.
[65] JAHANI-NEZHAD T, MADDAH-ALI M A, LI S, et al. SwiftAgg+: achieving asymptotically optimal communication loads in secure aggregation for federated learning[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(4): 977-989.
[66] RIVEST R L, ADLEMAN L, DERTOUZOS M L. On data banks and privacy homomorphisms[J]. Foundations of Secure Computation, 1978, 4(11): 169-180.
[67] ZHANG S, LI Z, CHEN Q, et al. Dubhe: towards data unbiasedness with homomorphic encryption in federated learning client selection[C]//Proceedings of the 50th International Conference on Parallel Processing, Lemont, Aug 9-12, 2021. New York: ACM, 2021: 1-10.
[68] PARK J, LIM H. Privacy-preserving federated learning using homomorphic encryption[J]. Applied Sciences, 2022, 12(2): 734.
[69] MA J, NAAS S A, SIGG S, et al. Privacy‐preserving federated learning based on multi‐key homomorphic encryption[J]. International Journal of Intelligent Systems, 2022, 37(9): 5880-5901.
[70] LIN Y, HAN S, MAO H, et al. Deep gradient compression: reducing the communication bandwidth for distributed training[J]. arXiv:1712.01887, 2017.
[71] HU R, GONG Y, GUO Y. Federated Learning with sparsification-amplified privacy and adaptive optimization[C]//Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, Montreal, Aug 19-27, 2021: 1463-1469.
[72] WAINAKH A, VENTOLA F, Mü?IG T, et al. User-level label leakage from gradients in federated learning[J]. Proceedings on Privacy Enhancing Technologies, 2022(2): 227-244.
[73] PILLUTLA K, KAKADE S M, HARCHAOUI Z. Robust aggregation for federated learning[J]. IEEE Transactions on Signal Processing, 2022, 70: 1142-1154.
[74] ASOODEH S, CHEN W N, CALMON F P, et al. Differentially private federated learning: An information-theoretic perspective[C]//Proceedings of the IEEE International Symposium on Information Theory (ISIT), Jul 12-20, 2021. Piscataway: IEEE, 2021: 344-349.
[75] LYU L J, YU H, MA X J, et al. Privacy and robustness in federated learning: attacks and defenses[J]. arXiv:2012. 06337, 2020.
[76] CANONNE C L, KAMATH G, STEINKE T. The discrete gaussian for differential privacy[C]//Advances in Neural Information Processing Systems, 2020: 15676-15688.
[77] KAIROUZ P, LIU Z, STEINKE T. The distributed discrete Gaussian mechanism for federated learning with secure aggregation[C]//Proceedings of the 38th International Conference on Machine Learning, Jul 18-24, 2021: 5201-5212.
[78] CHOUDHURY O, GKOULALAS-DIVANIS A, SALONIDIS T, et al. Differential privacy-enabled federated learning for sensitive health data[J]. arXiv:1910.02578, 2019.
[79] SHI Y, LIU Y, WEI K, et al. Make landscape flatter in differentially private federated learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Jun 18-22, 2023. Piscataway: IEEE, 2023: 24552-24562.
[80] LANG N, SOFER E, SHAKED T, et al. Joint privacy enhancement and quantization in federated learning[J]. IEEE Transactions on Signal Processing, 2023, 71: 295-310.
[81] SUI D, CHEN Y, ZHAO J, et al. Feded: federated learning via ensemble distillation for medical relation extraction[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , Nov 16-20, 2020: 2118-2128.
[82] ZHU Z, HONG J, ZHOU J. Data-free knowledge distillation for heterogeneous federated learning[C]//Proceedings of the 38th International Conference on Machine Learning, Jul 18-24, 2021: 12878-12889.
[83] ZHANG Z, WANG S, HONG Y, et al. Distributed dynamic map fusion via federated learning for intelligent networked vehicles[C]//Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Xi’an, May 30-Jun 5, 2021. Piscataway: IEEE, 2021: 953-959.
[84] KANG J, XIONG Z, JIANG C, et al. Scalable and communication?efficient decentralized federated edge learning with multi-blockchain framework[C]//Proceedings of the Second International Conference on Blockchain and Trustworthy System (BlockSys 2020), Dali, Aug 6-7, 2020. Berlin: Springer, 2020: 152-165.
[85] CHANG Y, FANG C, SUN W. A blockchain-based federated learning method for smart healthcare[C]//Proceedings of the IEEE Wireless Communications and Networking Conference Workshops (WCNCW), Jun 25, 2020.
[86] CHEN J, XUE J, WANG Y, et al. Privacy-preserving and traceable federated learning for data sharing in industrial IoT applications[J]. Expert Systems with Applications, 2023, 213: 119036.
[87] YUROCHKIN M, AGARWAL M, GHOSH S, et al. Bayesian nonparametric federated learning of neural networks[C]//Proceedings of the 36th International Conference on Machine Learning, Long Beach, Jun 9-15, 2019: 7252-7261.
[88] GUHA N, TALWALKAR A, SMITH V. One-shot federated learning[J]. arXiv:1902.11175, 2019.
[89] ZHOU Y, PU G, MA X, et al. Distilled one-shot federated learning[J]. arXiv:2009.07999, 2020.
[90] LI Q, HE B, SONG D. Practical one-shot federated learning for cross-silo setting[J]. arXiv:2010.01017, 2020.
[91] KASTURI A, ELLORE A R, HOTA C. Fusion learning: a one shot federated learning[C]//Proceedings of the 20th International Conference on Computational Science, Netherlands, Jun 3-5, 2020. Berlin: Springer, 2020: 424-436.
[92] XIONG Y, WANG R, CHENG M, et al. Feddm: iterative distribution matching for communication-efficient federated learning[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, Jun 18-22, 2023. Piscataway: IEEE, 2023: 16323-16332.
[93] LIU Y, KUMAR N, XIONG Z, et al. Communication-efficient federated learning for anomaly detection in industrial internet of things[C]//Proceedings of the 2020 IEEE Global Communications Conference (GLOBECOM 2020), Taiwan, China, Dec 7-11, 2020. Piscataway: IEEE, 2020: 1-6.
[94] LEE H, KIM J, AHN S, et al. Digestive neural networks: a novel defense strategy against inference attacks in federated learning[J]. Computers & Security, 2021, 109: 102378.
[95] 林伟伟, 石方, 曾岚, 等.联邦学习开源框架综述[J].计算机研究与发展, 2023(7): 1551-1580.
LIN W W, SHI F, ZENG L, et al. Survey of federated learning open-source frameworks[J]. Journal of Computer Research and Development, 2023(7): 1551-1580.