Design and Implementation of Intelligent Interference Avoidance System Based on NS3-gym Framework

doi:10.3778/j.issn.1002-8331.2111-0074

Abstract

Abstract: The agents need to interact with the environment to learn the interference features and optimize the anti-interference strategies, when the multi-agent wireless communication networks are subject to the external malicious interference. It is necessary to design an intelligent interference avoidance simulation platform in order to effectively simulate and verify the learning interaction between agent and external interference environment. An intelligent interference avoidance system based on NS3-gym framework is proposed. NS3 implements a network simulation scenario and shares the sensed network state data as the input of agent. Agent provides interference avoidance strategy via learning and analyzing the input data, then returns it to NS3 for anti-interference strategy deployment through the interaction between gym and NS3. NS3-gym framework provides an interface for information exchange between NS3 and OpenAI gym. The simulation platform is built under Ubuntu20.04 system. The performance of Q-learning algorithm and WoLF-PHC algorithm is verified in four interference scenarios：sweep interference, greedy random strategy interference, follow interference and random interference, respectively. The simulation results indicate that the proposed system architecture and the simulation platform are correct and efficient.

Key words: system simulation, interference avoidance, NS3-gym, reinforcement learning

摘要： 多智能体构建的无线通信网络在受到外部恶意干扰时，智能体需要与环境进行大量交互来学习干扰规律和优化抗干扰策略。为了有效模拟和验证智能体与外部干扰环境的学习交互过程，需要构建智能干扰规避仿真系统。提出了一种基于NS3-gym框架的智能干扰规避系统，NS3模拟智能通信网络场景并将感知到的网络状态数据作为智能体的输入，智能体对输入数据进行学习分析得到干扰规避决策，并通过gym与NS3之间的交互将其返回到NS3中的仿真网络进行抗干扰策略部署。NS3-gym框架提供了NS3和OpenAI gym之间进行信息交互的接口。在Ubuntu20.04系统下搭建了智能干扰规避系统的仿真平台，分别验证了Q学习算法以及WoLF-PHC算法在扫频干扰、贪婪随机策略干扰、跟随干扰、随机干扰四种场景下的抗干扰性能。仿真结果证明了所提系统架构与仿真平台的正确性和有效性。

关键词: 系统仿真, 干扰规避, NS3-gym, 强化学习

CHEN Haitao, GONG Guangwei, ZHANG Jiao, ZHAO Haitao, XIONG Jun, WEI Jibo, ZHAN Dechuan. Design and Implementation of Intelligent Interference Avoidance System Based on NS3-gym Framework[J]. Computer Engineering and Applications, 2023, 59(4): 252-260.

陈海涛, 龚广伟, 张姣, 赵海涛, 熊俊, 魏急波, 詹德川. 基于NS3-gym框架的智能干扰规避系统设计与实现[J]. 计算机工程与应用, 2023, 59(4): 252-260.

References

[1] ZHI R，ZHANG L，ZHENG Z.Cognitive frequency hopping[C]//2008 International Conference on Cognitive Radio Oriented Wireless Networks & Communications，2008：1-4.
[2] SEDAGHATNEJAD S，FARHANG M.Detectability of chaotic direct-sequence spread-spectrum signals[J].IEEE Wireless Communication Letters，2015，4（6）：589-592.
[3] NAN H，YOO S J.Interference avoidance throughput optimization in cognitive radio ad hoc networks[J].EURASIP Journal on Wireless Communications and Networking，2012（1）：295-297.
[4] 张建军，唐雅娟.无线局域网的速率自适应算法设计与仿真分析[J].计算机工程与应用，2011，47（4）：109-113.
ZHANG J J，TANG Y J.Design and simulation analysis of rate adaptation for WLANs[J].Computer Engineering and Applications，2011，47（4）：109-113.
[5] HAN H D，ZHU C X，VIOREL D，et al.Resource allocation and beamforming algorithm based on interference avoidance approach for device-to-device communication underlaying LTE cellular network[J].Communications & Network，2013，5（3）：367-373.
[6] 王艺，段哲民，ZAIN-UL-ABIDEEN，等.基于空时自适应处理的多波束抗干扰改进算法[J].计算机工程与应用，2013，49（17）：221-225.
WANG Y，DUAN Z M，ZAIN-UL-ABIDEEN，et al.Improved multi-beam anti-jam algorithm based on space-time adaptive processing[J].Computer Engineering and Applications，2013，49（17）：221-225.
[7] 黄家邦.基于机器学习的通信抗干扰技术研究[D].成都：电子科技大学，2020.
HUANG J B.Research on anti-jamming communication technology based on machine learning[D].Chengdu：University of Electronic Science and Technology of China，2020.
[8] 宋佰霖，许华，蒋磊，等.一种基于深度强化学习的通信抗干扰智能决策方法[J].西北工业大学学报，2021，39（3）：641-649.
SONG B L，XU H，JIANG L，et al.An intelligent decision-making method for anti-jamming communication based on deep reinforcement learning[J].Journal of Northwestern Polytechnical University，2021，39（3）：641-649.
[9] LI F，SHAO W，ZHOU Q，et al.Interference avoidance scheme based on reinforcement learning[C]//2020 IEEE 3rd International Conference on Information Systems and Computer Aided Education，2020：221-224.
[10] AREF M A，JAYAWEERA S K.Robust deep reinforcement learning for interference avoidance in wideband spectrum[C]//2019 IEEE Cognitive Communications for Aerospace Applications Workshop，2019：1-5.
[11] 茹新宇，刘渊.网络仿真器NS3的剖析与探究[J].计算机技术与发展，2018，28（3）：72-77.
RU X Y，LIU Y.Analysis and research on network simulator3[J].Computer Technology and Development，2018，28（3）：72-77.
[12] REZAZADEH F，CHERGUI H，ALONSO L，et al.Continuous multi-objective zero-touch network slicing via twin delayed DDPG and OpenAI Gym[C]//2020 IEEE Global Communications Conference，2020：1-6.
[13] LEHNERT H，ARAYA M，CARRASCO-DAVIS R，et al.Bio-inspired deep reinforcement learning for autonomous navigation of artificial agents[J].IEEE Latin America Transactions，2019，17（12）：2037-2044.
[14] TANNER B，WHITE A M.RL-Glue：language-independent software for reinforcement-learning experiments[J].Journal of Machine Learning Research，2009，19（6）：2133-2136.
[15] BOUGIE N，CHENG L K，ICHISE R.Combining deep reinforcement learning with prior knowledge and reasoning[J].ACM SIGAPP Applied Computing Review，2018，18（2）：33-45.
[16] GAW?OWICZ P，ZUBOW A.NS-3 meets OpenAI Gym：the playground for machine learning in networking research[C]//22nd International ACM Conference，2019：310-319.
[17] SLIMENI F，CHTOUROU Z，SCHEERS B，et al.Cooperative Q-learning based channel selection for cognitive radio networks[J].Wireless Networks，2018，25（4）：4161-4171.
[18] 韩晨，牛英滔.基于分层Q学习的联合抗干扰算法[J].计算机工程，2019，45（5）：279-284.
HAN C，NIU Y T.Joint anti-jamming algorithm based on hierarchical Q learning[J].Computer Engineering，2019，45（5）：279-284.
[19] YANG H，XIONG Z，ZHAO J，et al.Intelligent reflecting surface assisted anti-jamming communications：a fast reinforcement learning approach[J].IEEE Transactions on Wireless Communications，2020，20（3）：1963-1974.