Abstract: For the problem of the convolution neural network’s high classification accuracy but poor real-time performance in the image classification task, a distilling knowledge algorithm with scale factor is proposed. This method adds the scale factor to measure the close relationship between the sample classes based on the existing “distilling knowledge” algorithm, enrich the network compression means, which can make the convolution neural network perform “distilling knowledge” more accurately. The principle is to use the scale factor error as part of the cost function to adjust the neural parameters of the neural network, which makes the generalization ability of neural network more similar to the compression reference network with better classification performance ability. The results show that the neural network compression algorithm with scale factor can describe the inter-class close relation of the training set more detailly and have better training performance than the original “distilling knowledge” algorithm and then the neural network with better generalization performance and higher precision is trained. It is realized that a large degree of reduction in neural network classification time under the premise that network classification accuracy is reduced as little as possible in order to achieve the purpose of network compression.

Key words: convolution neural network, image classification, network model compression, distilling the knowledge, real-time

摘要： 针对卷积神经网络在图像分类任务中，分类准确率高但实时性差的问题。提出了一种含比例因子的“知识提取”算法。此方法在已有的“知识提取”算法上，加入了衡量样本类间相近关系的比例因子，充实了网络压缩手段，使得神经网络可以更精确地进行“知识提取”。其原理是将比例因子误差值作为代价函数的一部分参与训练调节神经网络的神经元参数，进而使得神经网络的泛化能力更加趋近于具有更好分类表现能力的压缩参考网络。结果表明，含比例因子的神经网络压缩算法可以更细致地刻画训练集的类间相近关系，拥有比原“知识提取”算法更好的训练性能，进而训练出泛化性能更强、精度更高的神经网络。实现了在网络分类准确率下降尽量小的前提下，较大程度地减少神经网络的分类耗时，以达到网络压缩的目的。

关键词: 卷积神经网络, 图像分类, 网络模型压缩, 知识提取, 实时性