Computer Engineering and Applications ›› 2015, Vol. 51 ›› Issue (5): 107-115.

Previous Articles     Next Articles

Research on bitmap join index service mechanism

ZHANG Yansong1,2, SU Mingchuan1, ZHANG Yu1, WANG Fangzhou1   

  1. 1.School of Information, Renmin University of China, Beijing 100872, China
    2.National Survey Research Center, Renmin University of China, Beijing 100872, China
  • Online:2015-03-01 Published:2015-04-08

位图连接索引服务机制研究

张延松1,2,苏明川1,张  宇1,王方舟1   

  1. 1.中国人民大学 信息学院,北京 100872
    2.中国人民大学 中国调查与数据中心,北京 100872

Abstract: Bitmap join index is an effective index mechanism for optimizing join performance between tables in data warehouse. In big data in-memory analytical processing scenario, bitmap join index has to consider tradeoffs between memory and CPU cost. Moreover, new processor platform should also be considered for its performance gains and data access latency. This paper proposes a service oriented bitmap join index mechanism, its main characteristics can be summarized as follows: independent of database as a self-management index mechanism, memory space constraint TOP K keywords bitmap join index mechanism, processor-conscious bitmap join index technique. The bitmap join index service removes the built-in bitmap join index from database as an outside bitmap join index service layer, the query workloads analysis and index management models change the traditional DBA manual management on index. Furthermore, bitmap join index service mechanism can effectively promote performance of bitmap join index service and make it more flexible. The experimental results show that bitmap join index service mechanism can effectively promote efficiency of index storage and access, with parallel processing power of GPGPU, the performance of bitmap join index service and the whole database query processing can be markedly promoted.

Key words: bitmap join index, General-Purpose Graphics Processing Unit(GPGPU), keyword bitmap join index, processor-conscious bitmap join index

摘要: 位图连接索引是数据仓库中一种有效的优化表间连接操作性能的索引机制。在大内存分析处理应用场景下,位图连接索引不仅需要权衡索引的内存和CPU开销,还需要进一步考虑处理器平台所带来的性能收益和数据访问延迟。提出了基于服务的位图连接索引管理机制,其主要特点体现在三个方面:独立于数据库的自管理索引机制;基于存储空间约束的TOP K关键字位图连接索引机制;处理器敏感(processor-conscious)的位图连接索引技术。索引服务将索引从数据库中内置的数据结构变成数据库外的索引服务层,通过对用户查询负载的分析模块和索引服务管理模块改变传统的由数据库管理员人工管理索引的模式,同时借助于协处理器和内存云技术提高索引服务的性能和灵活性。实验测试结果表明,索引服务机制能够有效地提高索引存储和访问效率,在通用GPU的强大并行处理能力的支持下,位图连接索引服务的性能和数据库整体查询处理性能都得到了显著的提升。

关键词: 位图连接索引, 通用图形处理器(GPGPU), 关键字位图连接索引, 处理器敏感位图连接索引