Abstract:

Smart cities, smart factories pose challenges to the performance and connectivity of Internet of Things（IoT）. The emergence of edge computing makes up for these limited capabilities. By migrating intensive computing tasks from them to Edge Nodes（EN）, IoT devices can save more energy while still maintaining their services quality. Computational offloading decisions involve collaboration and complex resource management, and they should determine it in real time based on dynamic workloads and network environments. This paper adopts the method of a simulation experiment. Maximize long-term utility by deploying deep reinforcement learning agents on both IoT devices and edge nodes, and introduce federated learning to distribute training deep reinforcement learning agents. First, build an IoT system that supports edge computing. IoT downloads existing models from EN to train; they offload intensive computing tasks to EN training; IoT uploads updated parameters to EN, and EN aggregates the parameters with the model at EN to get a new model. The cloud can get new models and aggregates at EN, and IoT can also get updated parameters from EN to apply to the device. After several iterations, IoT can get close to centralized training performance and reduce the transmission cost between IoT devices and edge nodes. Experiments verify the effectiveness of decision-making schemes and federated learning in dynamic IoT environment.

Key words: federated learning, computation offloading, Internet of Things（IoT）, edge computing

摘要： 智能城市、智慧工厂等对物联网设备（Internet of Things，IoT）的性能和连接性提出了挑战。边缘计算的出现弥补了这些能力受限的设备，通过将密集的计算任务从它们迁移到边缘节点（Edge Node，EN），物联网设备能够在节约更多能耗的同时，仍保持服务质量。计算卸载决策涉及协作和复杂的资源管理，应该根据动态工作负载和网络环境实时确定计算卸载决策。采用模拟实验的方法，通过在物联网设备和边缘节点上都部署深度强化学习代理来最大化长期效用，并引入联盟学习来分布式训练深度强化学习代理。首先构建支持边缘计算的物联网系统，IoT从EN处下载已有模型进行训练，密集型计算任务卸载至EN进行训练；IoT上传更新的参数至EN，EN聚合该参数与EN处的模型得到新的模型；云端可在EN处获得新的模型并聚合，IoT也可以从EN获得更新的参数应用在设备上。经过多次迭代，该IoT能获得接近集中式训练的性能，并且降低了物联网设备和边缘节点之间的传输成本，实验证实了决策方案和联盟学习在动态物联网环境中的有效性。

关键词: 联盟学习, 计算卸载, 物联网（IoT）, 边缘计算