Missing value imputation algorithm based on clustering and auto-encoder

Computer Engineering and Applications ›› 2015, Vol. 51 ›› Issue (18): 13-17.

Previous Articles Next Articles

Missing value imputation algorithm based on clustering and auto-encoder

BU Fanyu1，2, CHEN Zhikui1, ZHANG Qingchen1

1.School of Software Technology, Dalian University of Technology, Dalian, Liaoning 116620, China
2.College of Vocation, Inner Mongolia University of Finance and Economics, Huhhot 010010, China

Online:2015-09-15 Published:2015-10-13

基于聚类和自动编码机的缺失数据填充算法

卜范玉1，2，陈志奎1，张清辰1

1.大连理工大学软件学院，辽宁大连 116620
2.内蒙古财经大学职业学院，呼和浩特 010010

Abstract

Abstract: Existing algorithms are of low efficiency and effectiveness in imputing missing data. Aiming at this problem, the paper proposes a missing value imputation algorithm based on the CFS clustering and improved auto-encoder model. To cluster the incomplete data set, it improves the CFS clustering algorithm by introducing the partial distance strategy that is used to measure the distance between two objects with missing values. It uses the improved CFS algorithm to cluster the data set. The improved auto-encoder is used to estimate the missing values according to the clustering result. Experiments demonstrate that this proposed algorithm can impute the missing values effectively.

Key words: incomplete data, Clustering by Fast Search and find of density peaks（CFS）, auto-encoder, partial distance strategy

摘要： 当前的不完整数据处理算法填充缺失值时，精度低下。针对这个问题，提出一种基于CFS聚类和改进的自动编码模型的不完整数据填充算法。利用CFS聚类算法对不完整数据集进行聚类，对降噪自动编码模型进行改进，根据聚类结果，利用改进的自动编码模型对缺失数据进行填充。为了使得CFS聚类算法能够对不完整数据集进行聚类，提出一种部分距离策略，用于度量不完整数据对象之间的距离。实验结果表明提出的算法能够有效填充缺失数据。

关键词: 不完整数据, 快速密度聚类算法（CFS）, 自动编码机, 部分距离策略

BU Fanyu1，2, CHEN Zhikui1, ZHANG Qingchen1. Missing value imputation algorithm based on clustering and auto-encoder[J]. Computer Engineering and Applications, 2015, 51(18): 13-17.

卜范玉1，2，陈志奎1，张清辰1. 基于聚类和自动编码机的缺失数据填充算法[J]. 计算机工程与应用, 2015, 51(18): 13-17.

[1]	WU Wenlong, ZHOU Xi, WANG Yi, WANG Baoquan. WKAG：Fraud Detection Method for Imbalanced Medical Insurance Data [J]. Computer Engineering and Applications, 2021, 57(9): 247-254.
[2]	CHEN Hai, QIAN Fulan, CHEN Jie, ZHAO Shu, ZHANG Yanping. Rating Prediction Model Based on Variational Auto-Encoder [J]. Computer Engineering and Applications, 2021, 57(22): 153-159.
[3]	ZHANG Yang, LU Mingming, ZHENG Yiji, LI Haifeng. Student Grade Prediction Based on Graph Auto-Encoder Model [J]. Computer Engineering and Applications, 2021, 57(13): 251-257.
[4]	ZHENG Linwen, ZHOU Jinzhi, HUANG Jing. Application of Deep Sparse Auto-Encoders in ECG Feature Extraction [J]. Computer Engineering and Applications, 2021, 57(11): 156-161.
[5]	CHEN Tiantian, HE Xiuqing, GE Wenshuang, HE Juhou. Method and Application of Massive Online Collaborative Learning Grouping [J]. Computer Engineering and Applications, 2021, 57(1): 92-98.
[6]	WANG Yu, LIU Fan, WANG Fei. Autoencoder Based Sparse Representation for Single Sample Face Recognition [J]. Computer Engineering and Applications, 2021, 57(1): 168-172.
[7]	ZHANG Yuchen, ZHANG Renchuan, LIU Jing, WANG Yongwei. Network Security Situation Evaluation Using Deep Auto-Encoders Network [J]. Computer Engineering and Applications, 2020, 56(6): 92-98.
[8]	DU Xiaolei, CHEN Zhigang, XU Xu, ZHANG Nan. Method of Improved Deep Wavelet Auto-Encoder in Bearing Fault Diagnosis [J]. Computer Engineering and Applications, 2020, 56(5): 263-269.
[9]	QIU Ningjia, WANG Xiaoxia, WANG Peng, ZHOU Sicheng, WANG Yanchun. Research on Convolutional Neural Network Algorithm Combined with Transfer Learning Model [J]. Computer Engineering and Applications, 2020, 56(5): 43-48.
[10]	TANG Wei, ZHENG Yuan, PAN Hong, XU Jingjun. Roller Bearing Fault Diagnosis Based on Stacked Auto-encoder with Dynamic Learning Rate [J]. Computer Engineering and Applications, 2020, 56(20): 264-269.
[11]	CHEN Jian, LIU Ming, XIONG Peng, MENG Xianhui, YANG Lin. ECG Signal Denoising Based on Convolutional Auto-encoder Neural Network [J]. Computer Engineering and Applications, 2020, 56(16): 148-155.
[12]	XIA Guo’en, TANG Qi, ZHANG Xianquan. Improved Multi-layer Perceptron Applied to Customer Churn Prediction [J]. Computer Engineering and Applications, 2020, 56(14): 257-263.
[13]	ZHANG Wenjing, CHEN Zhiya, FENG Fenling, LI Wan. Application of Deep Auto-Encoder Based on Sparse Theory in Highway Accident Casualty Forecast [J]. Computer Engineering and Applications, 2019, 55(7): 241-247.
[14]	AN Jicun, LV Xin, JI Linya. Traffic Congestion Prediction Based on Spatial-Temporal Correlation with Incomplete Data [J]. Computer Engineering and Applications, 2019, 55(4): 96-100.
[15]	ZHAI Zhengli, LIANG Zhenming, ZHOU Wei, SUN Xia. Research Overview of Variational Auto-Encoders Models [J]. Computer Engineering and Applications, 2019, 55(3): 1-9.

Missing value imputation algorithm based on clustering and auto-encoder

基于聚类和自动编码机的缺失数据填充算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics