Abstract: As a popular research field in recent years, deep learning has great application prospects, but there are many problems such as over-fitting, under-fitting, hidden layer number and node number selection. Aiming at the over-fitting problem of deep belief network, based on the theory of compressed sensing and the mathematical properties of zero norm, a sparse deep belief network based on non-means Gaussian distribution function is constructed. The over-fitting problem is solved by adding a sparse regular term in the pre-training phase, further to improve the deep belief network training process. Using the ORL and MINIST two public data sets to verify and analyze the improved scheme, the results show that it has a greater improvement in sparsity and accuracy than the existing improved schemes.

Key words: Deep Belief Network（DBN）, sparsity, Gaussian distribution, compressed sensing, 0 norm

摘要： 深度学习作为近年热门研究领域，具有极大的应用前景，但存在过拟合、欠拟合、隐藏层数和节点数选取等诸多问题。针对深度置信网络存在的过拟合问题，借鉴压缩感知理论和零范数的数学性质，构建了一种基于无均值高斯分布函数的稀疏深度置信网络。通过在预训练阶段添加稀疏正则项，进一步改进深度置信网络训练过程的方法加以解决过拟合问题。利用ORL和MINIST两种公开数据集上对该改进方案进行验证分析，结果表明其比现有的改进方案在稀疏性和准确性上有较大提升。

关键词: 深度置信网络（DBN）, 稀疏性, 高斯分布, 压缩感知, 0范数