关键词: 大数据, 前馈神经网络, 增量式学习, 奇异值分解（SVD）

Abstract: Data are generating at extremely high speed in the era of big data, whose contents and features are in the dynamic changes. Thus, the learning algorithm for neural networks should not only be able to adapt new instances, but also preserve the prior knowledge. However, the feed-forward neural network trained by typically Back-Propagation（BP） algorithm is not incremental in nature. This paper proposes an incremental back-propagation model for training neural networks. The goal of incremental leaning is achieved by adjusting the parameters and structures of the feed-forward neural network. The parameters are incrementally adapted by optimizing an objective function. The network topology is adapted by increasing the number of hidden neurons only if the parameters adaption perturbs the prior knowledge severely. After updating the model, the Singular Value Decomposition（SVD） of the weight matrix is performed to remove the redundant connections of each newly added hidden unit. Experimental results demonstrate that the proposed model can adjust its parameters and structure depending on the requirement of the big data process in real time with preserving the prior knowledge as much as possible in evolving environments.

Key words:  big data, feed-forward neural networks, incremental learning, Singular Value Decomposition（SVD）