多智能体边缘计算任务卸载

doi:10.3778/j.issn.1002-8331.2010-0482

摘要/Abstract

摘要： 边缘计算技术的发展为计算密集型业务提供了一种全新的选择，低能耗、低时延、实时处理等词语不断被提及，任务卸载引起了众多学者的注意。任务在本地执行还是卸载到服务器上执行，以及卸载到哪一台服务器上执行成为必须要解决的问题。在多智能体环境中提出一种新的目标函数，并构建数学模型；建立马尔可夫决策过程，定义动作、状态空间以及奖励函数，通过深度强化学习DRQN优化任务卸载策略。仿真实验结果表明，DRQN在能耗、花费和时延上的综合表现优于随机卸载、DQN等算法，证明了提出算法的有效性和实效性。

关键词: 边缘计算, 任务卸载, 深度强化学习

Abstract: The development of edge computing technology provides a new choice for computing-intensive services. Words such as low energy consumption, low latency, and real-time processing are constantly being mentioned, and task offloading has attracted the attention of many scholars. Whether the task is executed locally or offloaded to the server, and on which server the offloading is executed has become a problem that must be solved. A new objective function is proposed in a multi-agent environment, and a mathematical model is constructed. Subsequently, the Markov decision process is established, the action space, state space and reward function are defined, and the task offloading strategy is optimized through deep reinforcement learning DRQN. The simulation experiment results show that the comprehensive performance of DRQN in terms of energy consumption, cost and delay is better than random offloading, DQN and other algorithms, which proves the effectiveness of the algorithm.

Key words: edge computing, task offloading, deep reinforcement learning

赵庶旭, 元琳, 张占平. 多智能体边缘计算任务卸载[J]. 计算机工程与应用, 2022, 58(6): 177-182.

ZHAO Shuxu, YUAN Lin, ZHANG Zhanping. Multi-agent Edge Computing Task Offloading[J]. Computer Engineering and Applications, 2022, 58(6): 177-182.

参考文献

[1] DINH T Q，TANG J H，QUANG D L，et al.Offloading in mobile edge computing：task allocation and computational frequency scaling[J].IEEE Transactions on Communications，2017，65（8）：3571-3584.
[2] 田辉，范绍帅，吕昕晨，等.面向5G需求的移动边缘计算[J].北京邮电大学学报，2017，40（2）：1-10.
TIAN H，FAN S S，LYU X C，et al.Mobile edge computing for 5G requirements[J].Journal of Beijing University of Posts and Telecommunications，2017，40（2）：1-10.
[3] BOUET M，CONAN V.Mobile edge computing resources optimization：ageo-clustering approach[J].IEEE Transactions on Network and Service Management，2018，15（2）：787-796.
[4] ZHANG K，MAO Y M，LENG S P，et al.Contract-theoretic approach for delay constrained offloading in vehicular edge computing networks[J].Mobile Networks and Applications，2019，24（3）：1003-1014.
[5] MAHADEV S.The emergence of edge computing[J].Computer，2017，50（1）：30-39.
[6] ABBAS N，ZHANG Y，TAHERKORDI A，et al.Mobile edge computing：a survey[J].IEEE Internet of Things Journal，2018，5（1）：450-465.
[7] 李子姝，谢人超，孙礼，等.移动边缘计算综述[J].电信科学，2018，34（1）：87-101.
LI Z S，XIE R C，SUN L，et al.Overview of mobile edge computing[J].Telecommunications Science，2018，34（1）：87-101.
[8] 吴启晖，吴伟.无人机辅助边缘计算的能量效率最大化算法设计[J].通信学报，2020，41（10）：15-24.
WU Q H，WU W.Algorithm design on energy efficiency maximization for UAV-assisted edge computing[J].Journal on Communications，2020，41（10）：15-24.
[9] 谢人超，廉晓飞，贾庆民，等.移动边缘计算卸载技术综述[J].通信学报，2018，39（11）：138-155.
XIE R C，LIAN X F，JIA Q M，et al.Survey on computation offloading in mobile edge computing[J].Journal on Communications，2018，39（11）：138-155.
[10] MASHHADI F，SALINAS MONROY A，BOZORGCHENANI A，et al.Optimal auction for delay and energy constrained task offloading in mobile edge computing[J].Computer Networks，2020，183.
[11] LIU S G，QU X Y，LI Z L，et al.Research on task offloading of mobile edge computing[J].Scientific Journal of Economics and Management Research，2020，2（8）.
[12] LIU J，MAO Y Y，ZHANG J，et al.Delay-optimal computation task scheduling for mobile-edge computing systems[C]//2016 IEEE International Symposium on Information Theory（ISIT）.Barcelona：IEEE，2016：1451-1455.
[13] 张艮山，刘旭宁.资源受限移动边缘计算任务拆分卸载调度决策[J].计算机应用与软件，2019，36（10）：268-273.
ZHANG G S，LIU X N.Tasks split and offloading scheduling decision in mobile edge computing with limited resources[J].Computer Applications and Software，2019，36（10）：268-273.
[14] LI J，LU T J.Deep neural network based computational resource allocation for mobile edge computing[C]//2018 IEEE Globecom Workshops.Abu Dhabi，United Arab Emirates：IEEE，2018：1-6.
[15] 王汝言，梁颖杰，崔亚平.车辆网络多平台卸载智能资源分配算法[J].电子与信息学报，2020，42（1）：263-270.
WANG R Y，LIANG Y J，CUI Y P.Intelligent resource allocation algorithm for multi-platform offloading in vehicular networks[J].Journal of Electronics & Information Technology，2020，42（1）：263-270.
[16] LIU C F，BENNIS M，VINCENT H.Latency and reliability-aware task offloading and resource allocation for mobile edge computing[C]//Proceedings of 2017 IEEE Globecom Workshops.New York：IEEE，2017.
[17] CHEN W W，WANG D，LI K Q.Multi-user multi-task computation offloading in green mobile edge cloud computing[J].IEEE Transactions on Services Computing，2019，12（5）：726-738.
[18] LIU L Q，CHANG Z，GUO X J，et al.Multi-objective optimization for computation offloading in fog computing[J].IEEE Internet of Things Journal，2018，5（1）：283-294.
[19] BUSONIU L，BABUSKA R，DE SCHUTTER B.A comprehensive survey of multiagent reinforcement learning[J].IEEE Transactions on Systems Man & Cybernetics，Part C，2008，38（2）：156-172.
[20] 刘全，翟建伟，章宗长，等.深度强化学习综述[J].计算机学报，2018，41（1）：1-27.
LIU Q，ZHAI J W，ZHANG Z Z，et al.A survey on deep reinforcement learning[J].Chinese Journal of Computers，2018，41（1）：1-27.
[21] 徐翔斌，李志鹏.强化学习在运筹学的应用：研究进展与展望[J].运筹与管理，2020，29（5）：227-239.
XU X B，LI Z P.Research progress and prospects for application of reinforcement learning in operations research[J].Operations Research and Management Science，2020，29（5）：227-239.
[22] 刘建伟，高峰，罗雄麟.基于值函数和策略梯度的深度强化学习综述[J].计算机学报，2019，42（6）：1406-1438.
LIU J W，GAO F，LUO X L.Survey of deep reinforcement learning based on value function and policy gradient[J].Chinese Journal of Computers，2019，42（6）：1406-1438.
[23] MNIH V，KAVUKCUOGLU K，SILVER D，et al.Human-level control through deep reinforcement learning[J].Nature，2015，518（7540）：529-533.