Abstract:

At present, in the application of the Internet of Things, a large number of acquisition terminals are used for sensing environment, positioning services, condition monitoring and so on. This terminals continuously upload data, which brings great challenges to remote server data collection and real-time processing while improving the efficiency of production. It utilizes the Netty network communication library to construct a high-performance network communication processing service and designs a communication protocol for the terminal and the server. Then, time-consuming operation in network communication service is stripped out and data are pushed into streaming message processing system Kafka. Kafka consumers are responsible for subsequent data persistence and real-time analysis. A data injection system supporting high concurrency and low latency is implemented. The experimental results show that the system can work normally and maintain a fast response speed under the condition of 10,000-level connections.

Key words: Internet of Things data, network communications, Netty, Kafka

摘要：

当前在物联网应用中，大量采集终端被用于感知环境、定位服务、状态监测等应用，并源源不断地上传数据，在提产助效的同时，给远程服务端数据收集及实时处理带来巨大挑战。利用Netty网络通信库构造高性能的收集端网络通信处理服务并设计出可供采集终端与收集端使用的通信协议，剥离出网络通信业务中耗时操作并将数据推送至流式消息处理系统Kafka中，再由Kafka消费者负责后续数据持久化、实时分析工作。由此实现一个支持高并发、低延迟的数据接入系统。实验结果表明，在万级别连接情况下，该系统能正常工作且保持较快的响应速度。

关键词: 物联网数据, 网络通信, Netty, Kafka