Advances in iterative MapReduce

Abstract

Abstract: Iterative computations are pervasive among big data processing, but the traditional MapReduce cannot explicitly support iterative computation. In recent years, researchers have extended and improved the original MapReduce, and have developed a number of iterative MapReduce to better support iterative computation for big data processing. A comprehensive review of iterative MapReduce programming framework is provided. These research achievements are described in detail. Their basic ideas are given. Their characteristics, advantages and disadvantages are analyzed for each framework, and some technologies that have been adopted in these frameworks are compared. Some promising development trends for future research of iterative MapReduce are pointed out.

Key words: MapReduce, iterative computation, iterative MapReduce, parallel programming model, big data processing

摘要： 迭代计算普遍存在于大数据处理中，而传统的MapReduce不能显式地支持迭代计算。近几年，研究者扩展和改进原始MapReduce，已开发了若干迭代式MapReduce以更好地为大数据处理而支持迭代计算。对迭代式MapReduce编程框架进行综合评述，较详细地阐述了这些研究成果，给出了它们各自的基本思想，并分析了它们各自的特点、优势和不足，且对比了它们所采用的一些技术。对迭代式MapReduce未来的发展趋势进行了展望。

关键词: MapReduce, 迭代计算, 迭代式MapReduce, 并行编程模型, 大数据处理

LI Jinzhong, TANG Pengjie, XIA Jiewu, TAN Yunlan. Advances in iterative MapReduce[J]. Computer Engineering and Applications, 2015, 51(12): 123-132.

李金忠，汤鹏杰，夏洁武，谭云兰. 迭代式MapReduce研究进展[J]. 计算机工程与应用, 2015, 51(12): 123-132.

[1]	CHEN Yuanwen. Application of MapReduce Technology in Problem of Material Transportation and Stowage [J]. Computer Engineering and Applications, 2021, 57(12): 273-278.
[2]	LIU Jun, LI Wei, WU Mengting, CHEN Qifeng. New Design of Image Parallel Processing Model Based on Hadoop Platform [J]. Computer Engineering and Applications, 2019, 55(6): 186-190.
[3]	JI Changqing1，2, XIAO Peng3, LIU Chang4, WANG Zumin2, XI Fang2, SHAO Yinbo1, LI Zeyu2. Mobile Medical Call Algorithms Based on Spatial kNN Query [J]. Computer Engineering and Applications, 2019, 55(2): 206-212.
[4]	WANG Dezheng1, ZHANG Yinong1, YANG Fan2. Implementation of parallel PLS algorithm of process monitoring using MapReduce [J]. Computer Engineering and Applications, 2018, 54(24): 61-65.
[5]	CHEN Wanghu, YU Maoyi, MA Shengjun. Training BP neural networks with MapReduce based on sample data slice disruptions [J]. Computer Engineering and Applications, 2018, 54(2): 137-143.
[6]	XIA Xiaoyun, ZHANG Renbin, XIE Rui, WANG Cong. MapReduce approach for defect inspection of TFT-LCD [J]. Computer Engineering and Applications, 2017, 53(5): 202-206.
[7]	WANG Yonggui1, ZHANG Yan1, YANG Dongdong2. Research on algorithm of community discovery of wireless city based on MapReduce [J]. Computer Engineering and Applications, 2017, 53(4): 106-112.
[8]	LUO Jun, LI Jinhua. LSHBMRPK-means algorithm and its application. Computer Engineering and Applications [J]. Computer Engineering and Applications, 2017, 53(21): 62-67.
[9]	CHEN Yanan, ZHU Xijun. Association analysis of TCM asthma medication combination based on Hadoop [J]. Computer Engineering and Applications, 2017, 53(13): 95-98.
[10]	LI Sanmiao, LI Longshu. Performance analysis of four methods for handling small files in Hadoop [J]. Computer Engineering and Applications, 2016, 52(9): 44-49.
[11]	XIONG Zheng1，2, WANG Jinming2, ZHENG Haiyan1，2, LI Kunming1, XU Lizhen2, CHONG Zhihong2. RDF pattern matching algorithm with sorted view [J]. Computer Engineering and Applications, 2016, 52(8): 62-69.
[12]	WANG Yonggui, WU Chao, DAI Wei. K-means algorithm of random sample based on MapReduce [J]. Computer Engineering and Applications, 2016, 52(8): 74-79.
[13]	KAN Wenxiao1，2，3, WU Wenjing1, Andrei Tsaregorodtsev3, CHEN Gang1. Research on key technology of supporting multi-application [J]. Computer Engineering and Applications, 2016, 52(23): 25-34.
[14]	ZHANG Hong1，2, WANG Xiaoming1, CAO Jie2, MA Yanhong3, GUO Yirong1, WANG Min1. Research on optimized MapReduce model of Hadoop cloud platform [J]. Computer Engineering and Applications, 2016, 52(22): 22-25.
[15]	SHU Xiaowei, YANG Geng, NA Haiyang. Research on parallel crypt inverted index [J]. Computer Engineering and Applications, 2016, 52(20): 14-19.

Advances in iterative MapReduce

迭代式MapReduce研究进展

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics