Distributed singular value decomposition least squares estimation algorithm

Computer Engineering and Applications ›› 2014, Vol. 50 ›› Issue (3): 6-12.

Previous Articles Next Articles

Distributed singular value decomposition least squares estimation algorithm

LI Fan1，2, JIN Minglu1, LIU Ji3

1.Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, Liaoning 116024, China
2.Network & Experimental Teaching Center, Xinjiang University of Finance and Economics, Urumqi 830012, China
3.School of Statistics and Information, Xinjiang University of Finance and Economics, Urumqi 830012, China

Online:2014-02-01 Published:2014-01-26

分布式奇异值分解最小平方估计算法

李繁1，2，金明录1，刘继3

1.大连理工大学电子信息与电气工程学部，辽宁大连 116024
2.新疆财经大学网络与实验教学中心，乌鲁木齐 830012
3.新疆财经大学统计与信息学院，乌鲁木齐 830012

Abstract

Abstract: Singular Value Decomposition（SVD） for solving least-squares estimation problem is studied. This paper proposes Iterative Divide and Merge algorithm（IDMSVD）, aims to improve the problem that singular value decomposition in the estimation of parameters is very time-consuming and memory space. Based on IDMSVD a distributed iterative split and merge algorithm（MRDSVD） is proposed, using Hadoop’s MapReduce platform to achieve. The experimental results show, IDMSVD can effectively improve the SVD least squares solution required run time and memory space consuming problem. MRDSVD algorithm can further improve the running time of IDMSVD.

Key words: matrix decomposition, Singular Value Decomposition（SVD）, least-squares solution, large-scale dataset, distributed

摘要： 对奇异值（SVD）分解求解最小平方估计的问题进行了研究。提出迭代式分割与合并的算法（IDMSVD），目的是改善奇异值分解在估计参数时非常耗费时间以及内存空间的问题。基于IDMSVD提出了分布式迭代式分割与合并算法（MRDSVD），使用Hadoop平台的MapReduce来实现，实验结果显示，IDMSVD可以有效改善SVD求最小平方解耗费运行时间与内存空间的问题，MRDSVD算法可进一步改善IDMSVD的运行时间。

关键词: 矩阵分解, 奇异值分解, 最小平方估计, 大型数据集, 分布式

LI Fan1，2, JIN Minglu1, LIU Ji3. Distributed singular value decomposition least squares estimation algorithm[J]. Computer Engineering and Applications, 2014, 50(3): 6-12.

李繁1，2，金明录1，刘继3. 分布式奇异值分解最小平方估计算法[J]. 计算机工程与应用, 2014, 50(3): 6-12.

[1]	JIANG Kui, QIU Yuandong, ZHENG Haocheng. ICMPv6 DDoS Attack Detection Method Based on Information Entropy and LSTM [J]. Computer Engineering and Applications, 2021, 57(21): 148-154.
[2]	WANG Hairui, XIANYU Jianchuan. Application of Distributed Generation Configuration Based on Improved Sparrow Search Algorithm [J]. Computer Engineering and Applications, 2021, 57(20): 245-252.
[3]	TIAN Yang, CHEN Zhigang, SONG Xinxia, LI Tianming. Overview of Blockchain Application in Supply Chain Management [J]. Computer Engineering and Applications, 2021, 57(19): 70-83.
[4]	ZHANG Jun, LIAO Xuehua, YU Xuling, LEI Meng. Research on Realizing Relational Database In-Memory Storage Model [J]. Computer Engineering and Applications, 2021, 57(19): 123-128.
[5]	WANG Yingbo, SUN Yongdi. GNN-Based Matrix Factorization Recommendation Algorithm [J]. Computer Engineering and Applications, 2021, 57(19): 129-134.
[6]	YANG Xinmin, DONG Hongbin, TAN Chengyu, ZHOU Wen. Dendritic Cell Model Using Singular Value Decomposition and Information Gain [J]. Computer Engineering and Applications, 2021, 57(15): 156-162.
[7]	XIANG Yixuan, JIANG He, PAN Pinchen, SUN Conghui. Study on [K]-means Clustering Algorithm of Quadratic Power Coupling [J]. Computer Engineering and Applications, 2021, 57(14): 95-102.
[8]	WANG Wentao, LI Shumei, TANG Jie, LYU Weilong. DDoS Attack Detection Method Based on Probability Graph Model and DNN [J]. Computer Engineering and Applications, 2021, 57(13): 108-115.
[9]	ZHOU Bifeng, LUO Yiping. Aperiodically Intermittent Neutralization Control of Distributed Parameter Systems with Delay [J]. Computer Engineering and Applications, 2020, 56(8): 220-225.
[10]	WEI Zhanchen, LIU Xiaoyu, HUANG Qiulan, SUN Gongxing. Research on Optimization for Iteration-Intensive Applications on Spark [J]. Computer Engineering and Applications, 2020, 56(23): 68-73.
[11]	LIU Zujun, HE Ming, MA Ziyu, GU Lingfeng. UAV Formation Control Method Based on Distributed Consistency [J]. Computer Engineering and Applications, 2020, 56(23): 146-152.
[12]	HU Xuelin, Hasan·Wumaier. Review on Recommender Algorithms Based on Distributed Representation Technology [J]. Computer Engineering and Applications, 2020, 56(22): 13-24.
[13]	CHENG Zhenjing, CHENG Yaodong, CHEN Gang, WANG Lu, LI Haibo, HU Qingbao. High Energy Physics Data Placement Strategy Based on Random Forest [J]. Computer Engineering and Applications, 2020, 56(21): 60-64.
[14]	SONG Guangxin, WANG Liping. Utlizing Matrix Completion Optimization Model on Dynamic Network Link Prediction [J]. Computer Engineering and Applications, 2020, 56(16): 37-44.
[15]	LIU Chunling, ZHANG Li. Recommendation Algorithm for Improving Asymmetric Similarity and Associated Regularization [J]. Computer Engineering and Applications, 2020, 56(16): 45-49.

Distributed singular value decomposition least squares estimation algorithm

分布式奇异值分解最小平方估计算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics