Computer Engineering and Applications ›› 2024, Vol. 60 ›› Issue (5): 1-16.DOI: 10.3778/j.issn.1002-8331.2306-0243
• Research Hotspots and Reviews • Previous Articles Next Articles
ZHANG Shiwen, CHEN Shuang, LIANG Wei, LI Renfa
Online:
2024-03-01
Published:
2024-03-01
张世文,陈双,梁伟,李仁发
ZHANG Shiwen, CHEN Shuang, LIANG Wei, LI Renfa. Survey on Attack Methods and Defense Mechanisms in Federated Learning[J]. Computer Engineering and Applications, 2024, 60(5): 1-16.
张世文, 陈双, 梁伟, 李仁发. 联邦学习中的攻击手段与防御机制研究综述[J]. 计算机工程与应用, 2024, 60(5): 1-16.
Add to citation manager EndNote|Ris|BibTeX
URL: http://cea.ceaj.org/EN/10.3778/j.issn.1002-8331.2306-0243
[1] 张思思, 高旭光, 滑文强. 基于聚类与人工神经网络的遥感图像信息提取方法[J]. 电子设计工程, 2020, 28(15): 106-109. ZHANG S S, GAO X G, HUA W Q. Remote sensing image information extraction method based on clustering and artificial neural network[J]. International Electronic Elements, 2020, 28(15): 106-109. [2] MCMAHAN H B, MOORE E, RAMAGE D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proceeding of the 20th International Conference on Artificial Intelligence and Statistics, Ft Lauderdale FL, April 20-22, 2017. USA: JMLR, 2017: 1273-1282. [3] JAGIELSKI M, OPREA A, BIGGIO B, et al. Manipulating machine learning: poisoning attacks and countermeasures for regression learning[C]//Proceeding of the 39th IEEE Symposium on Security and Privacy, San Francisco, May 21-23, 2018. NJ: IEEE, 2018: 19-35. [4] WANG Z B, SONG M K, ZHANG Z F, et al. Beyond inferring class representatives: user-level privacy leakage from federated learning[C]//Proceeding of the 38th Annual IEEE International Conference on Computer Communications, Paris, April 29-May 2, 2019. NJ: IEEE, 2019: 2512-2520. [5] HITAJ B, ATENIESE G, PEREZ-CRUZ F. Deep models under the GAN: information leakage from collaborative deep learning[C]//Proceeding of the 2017 ACM SIGSAC Conference on Computer and Communications Security, Dallas, TX, October, 2017. New York: ACM, 2017: 603-618. [6] YANG Q, LIU Y, CHEN T J, et al. Federated machine learning[J]. ACM Transactions on Intelligent Systems and Technology, 2019, 10(2): 1-19. [7] LI T, SAHU A K, TALWALKARA, et al. Federated learning: challenges, methods, and future directions[J]. IEEE Signal Processing Magazine, 2020, 37(3): 50-60. [8] KAIROZ P, MCMAHAN H B, AVENT B, at al. Advances and open problems in federated learning[J]. Foundations and Trends? in Machine Learning, 2021, 14(1/2): 1-210. [9] MCMAHAN H B, MOORE E, RAMAGE D, et al. Federated learning of deep networks using model averaging[J]. arXiv:1602.05629,2016. [10] 刘艺璇, 陈红, 刘宇涵, 等. 联邦学习中的隐私保护技术[J]. 软件学报, 2022, 33(3): 1057-1092. LIU Y X, CHEN H, LIU Y H, et al. Privacy-preserving techniques in federated learning[J]. Journal of Software, 2022, 33(3): 1057-1092. [11] PAN S J, YANG Q. A Survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): 1345-1359. [12] 何英哲, 胡兴波, 何锦雯, 等. 机器学习系统的隐私和安全问题综述[J]. 计算机研究与发展, 2019, 56(10): 2049-2070. HE Y Z, HU X B, HE J W, et al. Privacy and security issues in machine learning systems: a survey[J]. Journal of Computer Research and Development, 2019, 56(10): 2049-2070. [13] BIGGIO B, NELSON B, LASKOV P. Poisoning attacks against support vector machines[J]. arXiv:1206.6389,2012. [14] RUBINSTEIN B, NELSON B, LING H, et al. ANTIDOTE: understanding and defending against poisoning of anomaly detectors[C]//Proceeding of the 9th ACM SIGCOMM Conference on Internet measurement, Chicago, Nov 4-6, 2009. New York: ACM, 2009: 1-14. [15] MUOZ-GONZALEZ L, BIGGIO B, DEMONTIS A, et al. Towards poisoning of deep learning algorithms with back-gradient optimization[J]. ACM, 2017, 17: 27-38. [16] SUN G, CONG Y, DONG J, et al. Data poisoning attacks on federated machine learning[J]. IEEE Internet of Things Journal, 2022, 9(13): 11365-11375. [17] TOLPEGIN V, TRUEX S, GURSOY M E, et al. Data poisoning attacks against federated learning systems[C]//Proceeding of the ESORICS 2020, UK, September 14-18, 2020. Berlin: Springer, 2020: 480-501. [18] LI Z, WU X K, JIANG C J. Efficient poisoning attacks and defenses for unlabeled data in DDoS prediction of intelligent transportation systems[J]. Security and Safety, 2022, 1: 145-165. [19] ZHOU X C, XU M, WU Y M, et al. Deep model poisoning attack on federated learning[J]. Future Internet, 2021, 13(3): 73. [20] HOSSAIN M T, ISLAM S, BADSHA B et al. DeSMP: differential privacy-exploited stealthy model poisoning attacks in federated learning[C]//Proceeding of the 17th International Conference on Mobility, Sensing and Networking (MSN), Exeter, Dec 13-15, 2021. NJ: IEEE, 2021: 167-174. [21] CAO X, GONG N Z. MPAF: model poisoning attacks to federated learning based on fake clients[C]//Proceeding of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, LA, USA, June 19-20, 2022. NJ: IEEE, 2022. [22] ZHAO S, MA X, ZHENG X, et al. Clean-label backdoor attacks on video recognition models[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, June 13-19, 2020. NJ: IEEE, 2020: 14431-14440. [23] BHAGOJI A N, CHAKRABORTY S, MITTAL P, et al. Analyzing federated learning through an adversarial lens[C]//Proceedings of the 36th International Conference on Machine Learning, Long Beach, June 9-15, 2019. Germany: Statistics, 2019: 1467-5463. [24] SZEGEDY C, ZAREMBA W, SUTSKEVER I, et al. Intriguing properties of neural networks[C]//Proceeding of the 2nd International Conference on Learning Representations, Banff, Canada, Apr 14-16, 2014. [25] 张思思, 左信, 刘建伟. 深度学习中的对抗样本问题[J]. 计算机学报, 2019, 42(8): 1886-1904. ZHANG S S, ZUO X, LIU J W. the problem of the adversarial examples in deep learning[J]. Chinese Journal of Computers, 2019, 42(8): 1886-1904. [26] LING X, JI L, ZOU J, et al. DEEPSEC: a uniform platform for security analysis of deep learning model[C]//Proceeding of the 2019 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA, May 19-23, 2019. NJ: IEEE, 2019: 673-690. [27] PAPERNOT N, MCDANIEL P, GOODFELLOW I. Transferability in machine learning: from phenomena to black-box attacks using adversarial samples[J]. arXiv:1605.07277,2016. [28] ZHANG Y H, JIA R X, PEI H Z, et al. The secret revealer: generative model inversion attacks against deep neural networks[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, June 13-19, 2020. NJ: IEEE, 2020: 250-258. [29] REN H C, DENG J J, XIE X H. GRNN: generative regression neural network—a data leakage attack for federated learning[J]. ACM Transactions on Intelligent Systems and Technology, 2022, 13(4): 1-24. [30] 孔锐, 蔡佳纯, 黄钢. 基于生成对抗网络的对抗攻击防御模型[J/OL]. 自动化学报(2020-07-23). http://www.aas.net.cn/cn/article/doi/10.16383/j.aas.2020.c200033?viewType=HTML. KONG R, CAI J C, HUANG G. Defense to adversarial attack with generative adversarial network[J/OL]. Acta Automatica Sinica (2020-07-23). http://www.aas.net.cn/cn/article/doi/10.16383/j.aas.2020.c200033?viewType=HTML. [31] BARRENO M, NELSON B, SEARS R, et al. CAN machine learning be secure[C]//Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security, Taipei, China, March 21-24, 2006. New York: ACM Press, 2006. [32] LEE H, KIM J, HUSSA IN R, et al. On defensive neural networks against inference attack in federated learning[C]//Proceedings of 2021 IEEE International Conference on Communications, Seoul, Korea, June 14-23, 2021. NJ: IEEE, 2021: 1-6. [33] SHOKRI R, STRONATI M, SONG C, et al. Membership inference attacks against machine learning models[C]//Proceedings of IEEE Symposium on Security and Privacy (SP), San Jose, May 22-26, 2017. NJ: IEEE, 2017: 3-18. [34] AONO Y, HAYASHI T, PHONG L T, et al. Scalable and secure logistic regression via homomorphic encryption[C]//Proceedings of the 6th ACM Conference on Data and Application Security and Privacy, New Orleans Louisiana USA, March 9-11. New York: ACM, 2016: 142-144. [35] MELIS L, SONG C Z, CRISTOFARO E D, et al. Exploiting unintended feature leakage in collaborative learning[C]//Proceeding of the IEEE Symposium on Security and Privacy, San Francisco, May 19-23, 2019. NJ: IEEE, 2019: 691-706. [36] NASR M, SHOKRI R, HOUMANSADR A. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning[C]//Proceeding of the IEEE Symposium on Security and Privacy, San Francisco, May 19-23, 2019. NJ: IEEE, 2019. [37] CHEN J L, ZHANG J L, ZHAO Y C, et al. Beyond model-level membership privacy leakage: an adversarial approach in federated learning[C]//Proceedings of the 29th International Conference on Computer Communications and Networks, Honolulu, Aug 3-6, 2020. NJ: IEEE, 2020: 1-9. [38] ZHANG J W, ZHANG J L, CHEN J J, et al. GAN enhanced membership inference: A passive local attack in federated learning[C]//Proceedings of the IEEE International Conference on Communications, Dublin, June 7-11, 2020. NJ: IEEE, 2020: 1-6. [39] SONG M K, WANG Z B, ZHANG Z F, et al. Analyzing user-level privacy attack against federated learning[J]. IEEE Journal on Selected Areas in Communications, 2020, 38(10): 2430-2444. [40] ZHU L G, LIU Z J, HAN S. Deep leakage from gradients[C]//Proceeding of the 33rd International Conference on Neural Information Processing Systems, NY, December 8-14, 2019. Berlin: Springer, 2019: 14747-14756. [41] SHEN M, WANG H, ZHANG B, et al. Exploiting unintended property leakage in blockchain-assisted federated learning for intelligent edge computing[J]. IEEE Internet of Things Journal, 2021, 8(4): 2265-2275. [42] TRAMER F, ZHANG F, JUELS A, et al. Stealing machine learning models via prediction APIs[C]//Proceedings of the 25th USENIX Conference on Security Symposium, Austin, May 31, 2019. USA: USENIX Association, 2016: 601-618. [43] WANG B, GONG N Z. Stealing hyperparameters in machine learning[C]//Proceedings of 2018 IEEE Symposium on Security and Privacy (SP), San Francisco, May 20-24, 2018. NJ: IEEE, 2018: 36-52. [44] OH S J, AUGUSTIN M, SCHIELE B, et al. Towards reverse engineering black-box neural networks[J]. arXiv:1711.01768,2017. [45] FREDRIKSON M, JHA S, RISTENPART T. Model inversion attacks that exploit confidence information and basic counter measures[C]//Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, Denver, Colorado, October 12-16, 2015. New York: Association for Computing Machinery, 2015: 1322-1333. [46] ATENIESE G, MANCINI L V, SPOGNARDI A, et al. Hacking smart machines with smarter ones: how to extract meaningful data from machine learning classifiers[J]. International Journal of Security and Networks, 2015, 10(3): 137-150. [47] LYU L J, YU H, YANG Q. Threats to federated learning: a survey[J]. arXiv:2003.02133,2020. [48] BOUACIDA N, MOHAPATRA P. Vulnerabilities in federated learning[J]. IEEE Access, 2021, 9: 63229-63249. [49] MOTHUKURI V, PARIZIE M, POURIYEH S, et al. A survey on security and privacy of federated learning[J]. Future Generation Computer Systems, 2021, 115: 619-640. [50] DWORK C, MCSHERRY F, NISSIM K, et al. Calibrating noise to sensitivity in private data analysis[C]//Lecture Notes in Computer Science, NY, March 4-7, 2006. Berlin: Springer, 2006: 265-284. [51] BASSILY R, NISSIM K, STEMMER U, et al. Practical locally private heavy hitters[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, December 4-9, 2017. Red Hook, NY, USA: Curran Associates Inc, 2017: 2285-2293. [52] BITTAU A, ERLINGSSON L, MANIATIS P, et al. Prochlo: strong privacy for analytics in the crowd[J]. Journal of Machine Learning Research, 2020, 21(1): 1532-4435. [53] ERLINGSSON L, FELDMAN V, MIRONOV I, et al. Amplification by shuffling: from local to central differential privacy via anonymity[C]//Proceedings of the Thirtieth Annual ACM-SIAM Symposium on Discrete Algorithms, San Diego, Jan 6-9, 2019. USA: Society for Industrial and Applied Mathematics, 2019: 2468-2479. [54] CHEU A, SMITH A, ULLMAN J, et al. Distributed differential privacy via shuffling[J]. arXiv:1808.01394,2018. [55] AVENT B, KOROLOVA A, ZEBER D, et al. BLENDER: enabling local search with a hybrid differential privacy model[J]. Journal of Privacy and Confidentiality, 2017, 9(2): 2575-8527. [56] MCMAHAN HB, RAMAGE D, TALWAR K. Learning differentially private recurrent language models[J]. arXiv:1710. 06963,2017. [57] CHOUDHURY O, GKOULALAS-DIVANIS A, SALONIDIS T, et al. Differential privacy-enabled federated learning for sensitive health data[J]. arXiv:1910.02578,2019. [58] GEYER R C, KLEIN T, NABI M. Differentially private federated learning: a client level perspective[J]. arXiv:1712.07557,2017. [59] BHOWMICK A, DUCHI J, FREUDIGER J, et al. Protection against reconstruction and its applications in private federated learning[J]. arXiv:1812.00984,2018. [60] ABADI M, CHU A, GOODFELLOW I, et al. Deep learning with differential privacy[C]//Proceedings of the 2016 ACM SIGASC Conference on Computer and Communications Security, NY, October 24-28, 2016, NY: ACM, 2016: 308-318. [61] RIVEST R L, ADLEMAN L, DERTOUZOS M L. On data banks and privacy homomorphisms[J]. Foundations of Secure Computation, 1978, 4(11): 169-180. [62] BONEH D, GOH E J, NISSIM K. Evaluating 2-DNF formulas on ciphertexts[C]//Proceedings of 24th Annual International Conference on the Theory and Applications of Cryptographic Techniques, Aarhus, Denmark, May 22-26, 2005. Berlin: Springer, 2005: 325-341. [63] FAN J, VERCAUTEREN F. Somewhat practical fully homomorphic encryption[J]. IACR Cryptology Eprint Archive, 2012, 2012: 144. [64] BRAKERSKI Z, GENTRY C, VAIKUNTANATHAN V. (Leveled) fully homomorphic encryption without bootstrapping[J]. ACM Transactions on Computation Theory (TOCT), 2014, 6(3): 1-36. [65] CHEON J H, KIM A, KIM M, et al. Homomorphic encryption for arithmetic of approximate numbers[C]//Proceedings of 23rd International Conference on the Theory and Application of Cryptology and Information Security, Hong Kong, China, December 3-7, 2017. Berlin: Springer, 2017: 409-437. [66] MADI A, STAN O, MAYOUE A, et al. A secure federated learning framework using homomorphic encryption and verifiable computing[C]//Proceedings of 2021 Reconciling Data Analytics, Automation, Privacy, and Security: A Big Data Challenge, Hamilton, Ontario, May 18-19, 2021. NJ: IEEE, 2020: 1-8. [67] PHONG L T, AONO Y, HAYASHI T, et al. Privacy-preserving deep learning via additively homomorphic encryption[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(5): 1333-1345. [68] REYZIN L, SMITH A, YAKOUBOV S. Turning HATE into LOVE: compact homomorphic Ad Hoc threshold encryption for scalable MPC[C]//Proceedings of 5th International Symposium on Cyber Security Cryptography and Machine Learning, Be'er Sheva, July 8-9, 2021. Berlin: Springer, 2021: 361-378. [69] ROTH E, NOBLE D, FALK BH, et al. Honeycrisp: largescale differentially private aggregation without a trusted core[C]//Proceedings of the 27th ACM Symposium on Operating Systems Principles, Huntsville, Ontario, October 27-30, 2019. NY: ACM, 2019: 196-210. [70] SHAMIR A. How to share a secret[J]. ACM, 1979, 22(11): 612-613. [71] BLAKLEY G R. Safeguarding cryptographic keys[C]//Proceedings of International Workshop on Managing Requirements Knowledge (MARK), New York, June 4-7, 1979. NJ: IEEE, 1979: 313-318. [72] ASMUTH C, BLOOM J. A modular approach to key safeguarding[J]. IEEE Transactions on Information Theory, 1983, 29(2): 208-210. [73] BONAWITZ K, IVANOV V, KREUTER B, et al. Practical secure aggregation for federated learning on user-held data[J]. arXiv:1611.04482,2016. [74] HAN G, ZHANG T T, ZHANG Y H, et al. Verifiable and privacy preserving federated learning without fully trusted centers[J]. Journal of Ambient Intelligence and Humanized Computing, 2022, 13(3): 1431-1441. [75] FEREIDOONI H, MARCHAL S, MIETTINEN M, et al. SAFELearn: secure aggregation for private federated learning[C]//Proceedings of 2021 IEEE Security and Privacy Workshops (SPW), San Francisco, May 27, 2021. NJ: IEEE, 2021: 56-62. [76] BARACALDO N, CHEN B, LUDWIG H, et al. Mitigating poisoning attacks on machine learning models: a data provenance based approach[C]//Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, Dallas, Texas, November 3, 2017. NY: ACM, 2017: 103-110. [77] ANDREINA S, MARSON G A, MLLERING H, et al. BaFFLe: backdoor detection via feedback-based federated learning[C]//Proceedings of IEEE 41st International Conference on Distributed Computing Systems (ICDCS), DC, July 7-10, 2021. NJ: IEEE, 2021: 52-863. [78] MIYATA T, MAEDA S, KOYAMA M, et al. Virtual adversarial training: a regularization method for supervised and semi-supervised learning[J]. IEEE Transactions on Pattern Analysis And Machine Intelligence, 2018, 41(8): 1979-1993. [79] NAIR A K, RAJ E D, SAHOO J. A robust analysis of adversarial attacks on federated learning environments[J]. Computer Standards & Interfaces, 2023, 86: 103723. [80] LIANG B, LI H C, SU M Q, et al. Detecting adversarial image examples in deep neural networks with adaptive noise reduction[J]. IEEE Transactions on Dependable and Secure Computing, 2021, 18(1): 72-85. [81] SHAH D, DUBE P, CHAKRABORTY S, et al. Adversarial training in communication constrained federated learning[J]. arXiv:2103.01319,2021. [82] SO J, GULER B, AVESTIMEHR A S. Turbo-aggregate: breaking the quadratic aggregation barrier in secure federated learning[J]. IEEE Journal on Selected Areas in Information Theory, 2021, 2(1): 479-489. [83] CHEN Y, LUO F, LI T, et al. A training-integrity privacy-preserving federated learning scheme with trusted execution environment[J]. Information Sciences, 2020, 522: 69-79. |
[1] | DUAN Xinru, CHEN Guirong, CHEN Aiwang, CHEN Chen, JI Weifeng. Review of Research on Information Security in Federated Learning [J]. Computer Engineering and Applications, 2024, 60(3): 61-77. |
[2] | LI Fengyun, GUO Hao, BI Yuanguo, LI Yining. Real-Time Trajectory Privacy Protection Method Based on Path Obfuscation [J]. Computer Engineering and Applications, 2024, 60(2): 288-294. |
[3] | TAN Rongjie, HONG Zhiyong, YU Wenhua, ZENG Zhiqiang. Decentralized Federated Learning Strategy for Non-Independent and Identically Distributed Data [J]. Computer Engineering and Applications, 2023, 59(1): 269-277. |
[4] | LIANG Guangjun, XIN Jianfang, WANG Qun, NI Xueli, GUO Xiangmin, XIA Lingling. Survey of IoT Forensics [J]. Computer Engineering and Applications, 2022, 58(8): 12-32. |
[5] | WANG Mengting, WANG Wei, ZHANG Qiang, LIU Momeng. Vehicular Network Message Authentication Technology Based on Proxy Vehicle [J]. Computer Engineering and Applications, 2022, 58(5): 131-137. |
[6] | LIU Fasheng, SUN Qixuan, LI Jianghua. Fusion of Double Block Chain Credit Data Storage and Query Scheme [J]. Computer Engineering and Applications, 2022, 58(2): 123-128. |
[7] | WANG Haotian, ZHENG Dongyi, LIU Fang, XIAO Nong. Personalized Federated Anomaly Detection Method for Multivariate Time Series Data [J]. Computer Engineering and Applications, 2022, 58(11): 60-65. |
[8] | WEI Lifei, LI Mengsi, ZHANG Lei, CHEN Congcong, CHEN Yujiao, WANG Qin. Privacy-Preserving Linear Regression Algorithm Based on Secure Two-Party Computation [J]. Computer Engineering and Applications, 2021, 57(22): 139-146. |
[9] | HE Zhiming, XU Yida. Electronic Medical Record Sharing Scheme Based on Blockchain and Searchable Encryption [J]. Computer Engineering and Applications, 2021, 57(21): 140-147. |
[10] | LYU Xin, ZHAO Liancheng, YU Jiyuan, TAN Bin, ZENG Tao, CHEN Juan. Trajectory-Clustering Based Privacy Protection Method for Continuous Query in LBS [J]. Computer Engineering and Applications, 2021, 57(2): 104-112. |
[11] | XIE Yuqing, WANG Yuan, JIANG Ying, YANG Miao, WANG Yongli. Privacy Protection Method Facilitating Data Sharing for Grid Manufacturing Data Lake [J]. Computer Engineering and Applications, 2021, 57(2): 113-118. |
[12] | SONG Guochao, CHU Guanghui, WU Shaoxin. Location Privacy Protection Approach Based on Interval Region [J]. Computer Engineering and Applications, 2020, 56(8): 66-73. |
[13] | ZENG Haiyan, ZUO Kaizhong, WANG Yonglu, LIU Rui. Semantic Diversity Location Privacy Protection Method in Road Network Environment [J]. Computer Engineering and Applications, 2020, 56(7): 102-108. |
[14] | XU Bin, LIANG Xiaobing, SHEN Bo. Non-interactive Queries Differential Privacy Protection Model in Big Data Environment [J]. Computer Engineering and Applications, 2020, 56(7): 116-121. |
[15] | ZHANG Siliang, LING Jie, CHEN Jiahui. Traceable Blockchain Ledger Privacy Protection Scheme [J]. Computer Engineering and Applications, 2020, 56(23): 31-37. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||