基于DRL的飞行自组网自适应多模式路由算法

doi:10.3778/j.issn.1002-8331.2203-0602

摘要/Abstract

摘要： 针对传统飞行自组网协议自适应能力不强、大规模网络应用场景效果不佳的问题，提出了一种基于深度强化学习的多模式路由算法。该算法综合利用系统吞吐量、分组递交率和平均端到端时延等参数构建价值函数，通过智能体自动调节各个无人机的路由工作模式，将大型网络分解为主体网络和数个与之相连的小型异构网络，降低了系统复杂度，局部性能达到最优，提升了整个网络的性能。使用NS3仿真平台测试了算法和传统协议AODV、DSDV的性能指标。仿真结果表明，算法显著优于传统协议，且网络规模越大、负载越高则优势越明显，平均吞吐量提升了55.46%，分组递交率提升了39.85%，平均端到端时延降低了60.94%。

关键词: 飞行自组网, 深度强化学习, 自适应路由算法, 混合路由

Abstract: Aiming at the problems of weak adaptability of traditional flying ad hoc network protocols and poor effect in large-scale network application scenarios, a multi-mode routing algorithm based on deep reinforcement learning is proposed. The algorithm constructs the value function by comprehensively using the parameters such as system throughput, packet delivery rate and average end-to-end delay. The agent automatically adjusts the routing mode of each UAV, decomposes the large network into the main network and several small heterogeneous networks connected with it, reduces the system complexity, optimizes the local performance, and improves the performance of the whole network. The agent automatically adjusts the routing mode of each UAV, decomposes the large network into the main network and several small heterogeneous networks connected with it, reduces the system complexity, optimizes the local performance, and improves the performance of the whole network. Simulation results show that the algorithm is significantly better than the traditional protocol, and the larger the network scale and the higher the load, the more obvious the advantage is. The average throughput is increased by 55.46%, the packet delivery rate is increased by 39.85%, and the average end-to-end delay is reduced by 60.94%.

Key words: flying ad hoc network（FANET）, deep reinforcement learning（DRL）, adaptive routing algorithm, hybrid routing

黄凯, 邱修林, 殷俊, 杨余旺. 基于DRL的飞行自组网自适应多模式路由算法[J]. 计算机工程与应用, 2023, 59(14): 268-274.

HUANG Kai, QIU Xiulin, YIN Jun, YANG Yuwang. Adaptive Multi-Mode Routing Algorithm for FANET Based on Deep Reinforcement Learning[J]. Computer Engineering and Applications, 2023, 59(14): 268-274.

参考文献

[1] ARAFAT M Y，POUDEL S，MOH S.Medium access control protocols for flying ad hoc networks：a review[J].IEEE Sensors Journal，2021，21（4）：4097-4121.
[2] MALHOTRA A，KAUR S.A comprehensive review on recent advancements in routing protocols for flying ad hoc networks[J].Transactions on Emerging Telecommunications Technologies，2019，33（3）：e3688.
[3] CHANDAN R R，KUSHWAHA B S，MISHRA P K.Performance evaluation of AODV，DSDV，OLSR routing protocols using NS-3 simulator[J].International Journal of Computer Network and Information Security，2018，10（7）：59-65.
[4] LEONOV A V，LITVINOV G A.About applying AODV and OLSR routing protocols to relaying network scenario in FANET with mini-UAVs[C]//2018 XIV International Scientific-Technical Conference on Actual Problems of Electronics Instrument Engineering（APEIE），2018：220-228.
[5] 董强健，赵海涛，郑超轶，等.无人机自组网典型场景构建及路由协议性能分析[J].通信技术，2019，52（9）：2149-2155.
DONG Q J，ZHAO H T，ZHEN C Y，et al.Construction of typical scenarios and performance analysis of routing protocols for UAV ad hoc network[J].Communications Technology，2019，52（9）：2149-2155.
[6] 刘庆华，黄声培，叶金才，等.一种无人机自组网DSR协议优化方法[J].计算机工程与应用，2022，58（1）：128-133.
LIU Q H，HUANG S P，YE J C，et al.DSR protocol optimization method for UAV ad hoc network[J].Computer Engineering and Applications，2022，58（1）：128-133.
[7] 陈侃松，李豪科，阮玉龙，等.基于局部邻居节点和链路权值的改进AODV路由协议[J].软件学报，2021，32（4）：1186-1200.
CHEN K S，LI H K，RUAN Y L，et al.Improved AODV routing protocol based on local neighbor nodes and link weights[J].Journal of Software，2021，32（4）：1186-1200.
[8] 周长家，周建国.一种基于OLSR的无人机网络适用路由算法[J].计算机工程，2021，47（10）：174-179.
ZHOU C J，ZHOU J G.An OLSR-based routing algorithm for UAV networks[J].Computer Engineering，2021，47（10）：174-179.
[9] 周满元，周力为.基于不同源节点数目的AODV路由协议的性能比较研究[J].计算机工程与应用，2007，43（18）：94-96.
ZHOU M Y，ZHOU L W.Performance comparison of AODV routing based on different of numbers source node[J].Computer Engineering and Applications，2007（18）：94-96.
[10] ZHU Jiangzhang，DU Qingsong.Group adaptive hybrid routing algorithm based on group mobility in tactical MANET[C]//Proceedings of 2016 IEEE Information Technology，Networking，Electronic and Automation Control Conference（ITNEC 2016），2016：36-40.
[11] 牛晓光，崔莉，黄长城.移动自组织网络中基于优化分簇的混合路由协议[J].通信学报，2010，31（10）：58-67.
NIU X G，CUI L，HUANG C C.Hybrid optimal cluster routing protocol for mobile ad hoc network[J].Journal on Communications，2010，31（10）：58-67.
[12] HOEBEKE J，MOERMAN I，DEMEESTER P.Adaptive routing for mobile ad hoc networks[J].EURASIP Journal on Wireless Communications and Networking，2012，2012（1）：1-19.
[13] SHIN D J，KIM J J.Deep reinforcement learning-based network routing technology for data recovery in exa-scale cloud distributed clustering systems[J].Applied Sciences，2021，11（18）：8727.
[14] YE S，XU L，LI X.Vehicle-mounted self-organizing network routing algorithm based on deep reinforcement learning[J].Wireless Communications and Mobile Computing，2021，9：1-9.
[15] RILEY G F，HENDERSON T R.The ns-3 network simulator[M]//Modeling and tools for network simulation.Berlin，Heidelberg：Springer，2010：15-34.
[16] BROCKMAN G，CHEUNG V，PETTERSSON L，et al.OpenAI Gym[J].arXiv：1606.01540，2016.
[17] GAW?OWICZ P，ZUBOW A.ns3-gym：extending openai gym for networking research[J].arXiv：1810.03943，2018.