Multi-Server Collaborative Task Caching Strategy in Edge Computing

doi:10.3778/j.issn.1002-8331.2211-0238

Abstract

Abstract: Aiming at the contradiction between the limited computing and storage resources of the edge server and a large number of user task requests, this paper designs an edge computing task cache network architecture based on multi-server collaboration. In this architecture, the edge server can cache and execute user tasks in memory. The cached tasks are executed on the cloud. Combining the characteristics of time-varying user task requests and users in adjacent areas tending to request similar tasks, a multi-server task caching algorithm（MSAC） based on improved Soft Actor-Critic is proposed. The algorithm aims at minimizing the user’s average task execution delay. In order to avoid repeatedly choosing the same action and converge to a local optimum, a maximum entropy model is introduced to encourage the edge server to explore the optimal action. By designing an experience sharing mechanism, the task caching strategy is optimized by collecting and learning the experience of local edge servers and adjacent servers. Simulation results show that compared with the highest popularity algorithm, independent SAC algorithm, DQN algorithm, and genetic algorithm, the proposed MSAC algorithm has the best effect in reducing the average execution delay of user tasks.

Key words: edge computing, task caching, multi-server collaboration, deep reinforcement learning

摘要： 针对边缘服务器有限的计算、存储资源与大量用户任务请求之间的矛盾，设计了基于多服务器协同的边缘计算任务缓存网络架构，该架构中边缘服务器可以在内存中缓存并执行用户任务，未缓存的任务放在云端执行。结合用户任务请求时变和邻近区域用户更倾向于请求相似任务的特点，提出一种基于改进Soft Actor-Critic的多服务器协同任务缓存算法（MSAC）。该算法以最小化用户平均任务执行时延为目标，为避免反复选择同一动作而收敛于局部最优，引入最大熵模型来鼓励边缘服务器探索最优动作。通过设计经验共享机制，收集并学习本地边缘服务器和相邻服务器的经验以优化任务缓存策略。仿真结果表明，与最高流行度算法、独立SAC算法、DQN算法、遗传算法相比，所提出的MSAC算法在降低用户任务平均执行时延方面的效果最好。

关键词: 边缘计算, 任务缓存, 多服务器协作, 深度强化学习

MA Shixiong, GE Haibo, SONG Xing. Multi-Server Collaborative Task Caching Strategy in Edge Computing[J]. Computer Engineering and Applications, 2023, 59(20): 245-253.

马世雄, 葛海波, 宋兴. 边缘计算中多服务器协同任务缓存策略[J]. 计算机工程与应用, 2023, 59(20): 245-253.

References

[1] SHI W，CAO J，ZHANG Q，et al.Edge computing：vision and challenges[J].Internet of Things Journal，2016，3（5）：637-646.
[2] 郑逢斌，朱东伟，臧文乾，等.边缘计算：新型计算范式综述与应用研究[J].计算机科学与探索，2020，14（4）：541-553.
ZHENG F B，ZHU D W，ZANG W Q，et al.Edge computing：review and application research on new computing paradigm[J].Journal of Frontiers of Computer Science and Technology，2020，14（4）：541-553.
[3] 张秋平，孙胜，刘敏，等.面向多边缘设备协作的任务卸载和服务缓存在线联合优化机制[J].计算机研究与发展，2021，58（6）：1318-1339.
ZHANG Q P，SUN S，LIU M，et al.Online joint optimization mechanism of task offloading and service cashing for multiedge device collaboration[J].Journal of Computer Research and Development，2021，58（6）：1318-1339.
[4] PHAM X Q，NGUYEN T D，Nguyen V D，et al.Joint service caching and task offloading in multi-access edge computing：a QoE-based utility optimization approach[J].IEEE Communications Letters，2020，25（3）：965-969.
[5] 张开元，桂小林，任德旺，等.移动边缘网络中计算迁移与内容缓存研究综述[J].软件学报，2019，30（8）：2491-2516.
ZHANG K Y，GUI X L，REN D W，et al.Survey on computation offloading and content caching in mobile edge networks[J].Journal of Software，2019，30（8）：2491-2516.
[6] MASHAEL K，ABDULLAH A，SOLTAN A，et al.Multilevel service-provisioning-based autonomous vehicle applications[J].Sustainability，2020，12（6）：1-16.
[7] XU J，CHEN L，ZHOU P.Joint service caching and task offloading for mobile edge computing in dense networks[C]//IEEE Conference on Computer Communications，Hawaii State，2018：207-215.
[8] CHEN W，HAN L.Time-efficient task caching strategy for multi-server mobile edge cloud computing[C]//IEEE 21st International Conference on High Performance Computing and Communications，Zhangjiajie，2019：1429-1436.
[9] LI C，XUE Y，WANG J，et al.Edge-oriented computing paradigms：a survey on architecture design and system management[J].ACM Computing Surveys，2018，51（2）：1-34.
[10] HAO Y，CHEN M，HU L，et al.Energy efficient task caching and offloading for mobile edge computing[J].IEEE Access，2018：11365-11373.
[11] MIN C，HAO Y，LONG H，et al.Green and mobility-aware caching in 5G networks[J].IEEE Transactions on Wireless Communications，2017，16（12）：8347-8361.
[12] TANG L，TANG B，KANG L，et al.A novel task caching and migration strategy in multi-access edge computing based on the genetic algorithm[J].Future Internet，2019，11（8）：181.
[13] CHEN L，JIE X，REN S，et al.Spatio-temporal edge service placement：a bandit learning approach[J].IEEE Transactions on Wireless Communications，2018，17（12）：8388-8401.
[14] MIAO Y，HAO Y，GHARAVI H，et al.Intelligent task caching in edge cloud via bandit learning[J].IEEE Transactions on Network Science and Engineering，2020，8（1）：625-637.
[15] SADEGHI A，SHEIKHOLESLAMI F，GIANNAKIS G B.Optimal and scalable caching for 5G using reinforcement learning of space-time popularities[J].IEEE Journal of Selected Topics in Signal Processing，2017，12（1）：180-190.
[16] ELGENDY I A，ZHANG W Z，HE H，et al.Joint computation offloading and task caching for multi-user and multi-task MEC systems：reinforcement learning-based algorithms[J].Wireless Networks，2021：2023-2038.
[17] POULARAKIS K，LLORCA J，TULINO A M，et al.Joint service placement and request routing in multi-cell mobile edge computing networks[C]//IEEE Conference on Computer Communications，Paris，2019：10-18.
[18] ZHONG S，GUO S，YU H，et al.Cooperative service caching and computation offloading in multi-access edge computing[J].Computer Networks，2021，189（5）：107916.
[19] FERIANI A，HOSSAIN E.Single and multi-agent deep reinforcement learning for AI-enabled wireless networks：a tutorial[J].IEEE Communications Surveys & Tutorials，2021，23（2）：1226-1252.
[20] WANG J，ZHAO L，LIU J，et al.Smart resource allocation for mobile edge computing：a deep reinforcement learning approach[J].IEEE Transactions on Emerging Topics in Computing，2019，9（3）：1529-1541.
[21] ZHU H，CAO Y，WANG W，et al.Deep reinforcement learning for mobile edge caching：review，new features，and open issues[J].IEEE Network，2018，32（6）：50-57.
[22] 刘雷，陈晨，冯杰，等.车载边缘计算中任务卸载和服务缓存的联合智能优化[J].通信学报，2021，42（1）：18-26.
LIU L，CHEN C，FENG J，et al.Joint intelligent optimization of task offloading and service caching for vehicu-
lar edge computing[J].Journal on Communications，2021，42（1）：18-26.
[23] YAO X Y，CHEN N J，YUAN X M，et al.Performance optimization of serverless edge computing function offloading based on deep reinforcement learning[J].Future Gene-
ration Computer Systems，2023，139：74-86.
[24] ZHOU H，JIANG K，LIU X，et al.Deep reinforcement learning for energy-efficient computation offloading in mobile-edge computing[J].IEEE Internet of Things Journal，2022，9（2）：1517-1530.
[25] 胡恒，金凤林，郎思琪.移动边缘计算环境中的计算卸载技术研究综述[J].计算机工程与应用，2021，57（14）：60-74.
HU H，JIN F L，LANG S Q.Review of computing offloading technology in mobile edge computing environment[J].Computer Engineering and Applications，2021，57（14）：60-74.
[26] WANG K，WANG X F，LIU X.A high reliable computing offloading strategy using deep reinforcement learning for IoVs in edge computing[J].Journal of Grid Computing，2021，19（2）：1-15.