面向时空资源分配的智能集群自组织预测方法

doi:10.3778/j.issn.1002-8331.2409-0393

摘要/Abstract

摘要： 在智能集群系统中由于动态环境和不完全信息的影响，资源的需求和供应会不断变化，需要实时调整资源分配策略。针对智能集群在不确定或有限信息下的动态资源分配问题，提出了一种面向时空资源分配的智能集群自组织预测方法。引入时空多智能体强化学习（STMARL）框架，通过深度Q网络（DQN）和长短期记忆单元（LSTM）对时空依赖性进行建模，分析智能集群在空间和时间上的动态行为和交互关系，进而使用一种基于信任模型的复合自组织Q-learning算法，使智能体在选择合作伙伴时能够利用自己的经验和其他智能体的意见。Q-learning算法使智能体能够独立评估适应关系的奖励，旨在提高系统的通信协调能力和动态环境适应能力以实现资源的合理分配。在无人电动汽车充电实验场景中，充电桩作为智能体，自组织充电调度在家庭和公共充电模式下，总负荷范围缩减幅度最高可达90.37%和49.33%，使总峰谷负荷差最小，有助于提高微电网的安全性和可靠性。

关键词: 时空建模, 多智能体, 自组织, 智能集群, 无人电动汽车充电场景

Abstract: In intelligent cluster systems, due to the impact of dynamic environments and incomplete information, the demand and supply of resources constantly change, requiring real-time adjustment of resource allocation strategies. To address the dynamic resource allocation problem of intelligent clusters under uncertain or limited information, a self-organizing prediction method for intelligent clusters oriented towards spatiotemporal resource allocation is proposed. This method introduces the spatio-temporal multi-agent reinforcement learning (STMARL) framework, which models spatio-temporal dependencies through deep Q-network (DQN) and long short-term memory unit (LSTM), analyzes the dynamic behavior and interaction relationships of intelligent clusters in space and time, and then uses a composite self-organizing Q-learning algorithm based on a trust model to enable agents to leverage their own experience and the opinions of other agents when selecting partners. The Q-learning algorithm enables the agent to independently evaluate the rewards associated with adaptation, aiming to improve the system’s communication coordination capabilities and dynamic environmental adaptability to achieve rational resource allocation. In the experimental scenario of unmanned electric vehicle charging, the charging pile, as an agent, can reduce the total load range by up to 90.37% and 49.33% in the home and public charging modes through self-organized charging scheduling, minimizing the total peak-valley load difference and helping to improve the safety and reliability of the microgrid.

Key words: spatiotemporal modeling, multi-agent, self-organization, intelligent clusters, unmanned electric vehicle charging scenario

王宏为, 刘玮, 郭靠, 王紫昊. 面向时空资源分配的智能集群自组织预测方法[J]. 计算机工程与应用, 2025, 61(14): 176-184.

WANG Hongwei, LIU Wei, GUO Kao, WANG Zihao. Intelligent Cluster Self-Organization Prediction Methods for Spatiotemporal Resource Allocation[J]. Computer Engineering and Applications, 2025, 61(14): 176-184.

参考文献

[1] 武文亮, 周兴社, 沈博, 等. 集群机器人系统特性评价研究综述[J]. 自动化学报, 2022, 48(5): 1153-1172.
WU W L, ZHOU X D, SHEN B, et al. A review of swarm robotic systems property evaluation research[J]. Acta Automatica Sinica, 2022, 48(5): 1153-1172.
[2] 孙彧, 曹雷, 陈希亮, 等. 多智能体深度强化学习研究综述[J]. 计算机工程与应用, 2020, 56(5): 13-24.
SUN Y, CAO L, CHEN X L, et al. Overview of multi-agent deep reinforcement learning[J]. Computer Engineering and Applications, 2020, 56(5): 13-24.
[3] YE D Y, ZHANG M J, ATHANASIOS V, et al. A survey of self-organization mechanisms in multiagent systems[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017, 47(3): 441-461.
[4] 陆军, 王飞跃, 董琦, 等. 多智能体系统的体系化和组织化博弈[J]. Frontiers of Information Technology Electronic Engineering, 2022, 23(7): 991-997.
LU J, WANG F Y, DONG Q, et al. Institutionalized and systematized gaming for multi-agent systems[J]. Frontiers of Information Technology Electronic Engineering, 2022, 23(7): 991-997.
[5] NIU H Y, WEI J M, CHEN Y Q. Optimal randomness for stochastic configuration network (SCN) with heavy-tailed distributions[J]. Entropy, 2021, 23(1): 56.
[6] BRUST M R, STRIMBU B M. A networked swarm model for UAV deployment in the assessment of forest environments[C]//Proceedings of the IEEE 10th International Conference on Intelligent Sensors, Sensor Networks and Information Processing, 2015: 1-6.
[7] WEI Z C, LIU F, YU Z W, et al. Reinforcement learning for a novel mobile charging strategy in wireless rechargeable sensor networks[C]//Proceedings of the 13th International Conference on Wireless Algorithms, Systems, and Applications, 2018: 485-496.
[8] CAO X, XU W, LIU X, et al. A deep reinforcement learning-based on-demand charging algorithm for wireless rechargeable sensor networks[J]. Ad Hoc Networks, 2021, 110: 102278.
[9] XU W Z, LIANG W F, LIN X, et al. Efficient scheduling of multiple mobile chargers for wireless sensor networks[J]. IEEE Transactions on Vehicular Technology, 2016, 65(9): 7670-7683.
[10] WANG K, WANG L, LIN C, et al. Prolonging lifetime for wireless rechargeable sensor networks through sleeping and charging scheduling[J]. International Journal of Communication Systems, 2020, 33(8): e4355.
[11] HAN G J, GUAN H F, WU J W, et al. An uneven cluster-based mobile charging algorithm for wireless rechargeable sensor networks[J]. IEEE Systems Journal, 2019, 13(4): 3747-3758.
[12] SONI S, SHRIVASTAVA M. Novel wireless charging algorithms to charge mobile wireless sensor network by using reinforcement learning[J]. SN Applied Sciences, 2019, 1(9): 1052.
[13] XU L, CHEN N C, CEHN Z Q, et al. Spatiotemporal forecasting in earth system science: methods, uncertainties, predictability and future directions[J]. Earth-Science Reviews 2021, 222: 103828.
[14] 田文, 杨帆, 尹嘉男, 等. 航路时空资源分配的多目标优化方法[J]. 交通运输工程学报, 2020, 20(6): 218-226.
TIAN W, YANG F, YIN J N, et al. Multi-objective optimization method of air route space-time resources allocation[J]. Journal of Traffic and Transportation Engineering, 2020, 20(6): 218-226.
[15] RAY D, VOHRA R. Coalition formation[J]. Handbook of Game Theory with Economic Applications, 2015, 4: 239-326.
[16] WI H, OH S, MUN J, et al. A team formation model based on knowledge and collaboration[J]. Expert Systems with Applications, 2009, 36(5): 9121-9134.
[17] INIGUEZ G, KERTESZ J, KASKI K K, et al. Opinion and community formation in coevolving networks[J]. Physical Review E, 2009, 80(6): 066119.
[18] DULAC-ARNOLD G, LEVINE N, MANKOWITZ D J, et al. Challenges of real-world reinforcement learning: definitions, benchmarks and analysis[J]. Machine Learning, 2021, 110(9): 2419-2468.
[19] 张婷婷, 宋爱国, 蓝羽石. 集群无人系统自适应结构建模与预测[J]. 中国科学: 信息科学, 2020, 50(3): 347-362.
ZHANG T T, SONG A G, LAN Y S. Adaptive structure modeling and prediction for swarm unmanned system[J]. SCIENTIA SINICA Informationis, 2020, 50(3): 347-362.
[20] ZENG X, PENG H, LI A. Effective and stable role-based multi-agent collaboration by structural information principles[C]//Proceedings of the AAAI Conference on Artificial Intelligence, 2023: 11772-11780.
[21] LIU Z J, ZHANG Y Z, LI P, et al. Dynamic LLM-agent network: an LLM-agent collaboration framework with agent team optimization[J]. arXiv:2310.02170, 2023.
[22] HONG S, ZHENG X, CHEN J, et al. MetaGPT: meta programming for multi-agent collaborative framework[J]. arXiv: 2308.00352, 2023.
[23] CHEN W, SU Y, ZUO J, et al. AgentVerse: facilitating multi-agent collaboration and exploring emergent behaviors in agents[J]. arXiv:2308.10848 , 2023.
[24] CHEN G, DONG S, SHU Y, et al. AutoAgents: a framework for automatic agent generation[J]. arXiv:2309.17288, 2023.