计算机工程与应用 ›› 2014, Vol. 50 ›› Issue (3): 6-12.

• 博士论坛 • 上一篇    下一篇

分布式奇异值分解最小平方估计算法

李  繁1,2,金明录1,刘  继3   

  1. 1.大连理工大学 电子信息与电气工程学部,辽宁 大连 116024
    2.新疆财经大学 网络与实验教学中心,乌鲁木齐 830012
    3.新疆财经大学 统计与信息学院,乌鲁木齐 830012
  • 出版日期:2014-02-01 发布日期:2014-01-26

Distributed singular value decomposition least squares estimation algorithm

LI Fan1,2, JIN Minglu1, LIU Ji3   

  1. 1.Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, Liaoning 116024, China
    2.Network & Experimental Teaching Center, Xinjiang University of Finance and Economics, Urumqi 830012, China
    3.School of Statistics and Information, Xinjiang University of Finance and Economics, Urumqi 830012, China
  • Online:2014-02-01 Published:2014-01-26

摘要: 对奇异值(SVD)分解求解最小平方估计的问题进行了研究。提出迭代式分割与合并的算法(IDMSVD),目的是改善奇异值分解在估计参数时非常耗费时间以及内存空间的问题。基于IDMSVD提出了分布式迭代式分割与合并算法(MRDSVD),使用Hadoop平台的MapReduce来实现,实验结果显示,IDMSVD可以有效改善SVD求最小平方解耗费运行时间与内存空间的问题,MRDSVD算法可进一步改善IDMSVD的运行时间。

关键词: 矩阵分解, 奇异值分解, 最小平方估计, 大型数据集, 分布式

Abstract: Singular Value Decomposition(SVD) for solving least-squares estimation problem is studied. This paper proposes Iterative Divide and Merge algorithm(IDMSVD), aims to improve the problem that singular value decomposition in the estimation of parameters is very time-consuming and memory space. Based on IDMSVD a distributed iterative split and merge algorithm(MRDSVD) is proposed, using Hadoop’s MapReduce platform to achieve. The experimental results show, IDMSVD can effectively improve the SVD least squares solution required run time and memory space consuming problem. MRDSVD algorithm can further improve the running time of IDMSVD.

Key words: matrix decomposition, Singular Value Decomposition(SVD), least-squares solution, large-scale dataset, distributed