Abstract: As modern computer systems continue to grow in scale and complexity, the number of its running log is also increasing. Due to the huge size of the log data, and the content is messy, the user has some difficulties in viewing the log data. Therefore, it is of great significance to effectively collect and process the log data of application running log. This paper proposes a framework for collection and service processing of application running log, using distributed collection strategy to collect log data, defining a multi-level data storage structure to store log data, and providing the user with log data query services. It analyzes the log data of an integrated disaster reduction system. It is used to verify the practicability of the proposed framework, and it is compared with the traditional log collection and analysis software ELK and Chukwa.

Key words: log data, framework, collection and processing, distributed collection, multi-level data storage, query service

摘要： 随着现代计算机系统的规模和复杂性不断增大，其运行日志的数量也不断增多，但是由于日志数据的规模巨大、内容杂乱，用户查看日志数据具有一定的困难，因此对应用软件运行日志进行有效收集与处理具有重要意义。提出了一种应用软件运行日志的收集与服务处理框架，利用分布式收集策略对日志数据收集，定义了一种多层次数据存储结构对日志数据进行存储，并向用户提供日志数据查询服务。通过对某综合减灾系统的日志数据分析验证了提出的日志数据收集与服务处理框架的实用性，并与传统的日志收集分析软件ELK和Chukwa进行了对比。

关键词: 日志数据, 框架, 收集处理, 分布式收集, 多层次数据存储, 查询服务