Computer Engineering and Applications ›› 2021, Vol. 57 ›› Issue (22): 86-91.DOI: 10.3778/j.issn.1002-8331.2011-0239

• Theory, Research and Development • Previous Articles     Next Articles

Neural Network Optimization Method Based on Invalid Filters Weight Regression

GU Shanghang, ZHANG Lijun, GUO Yuechao, XU Yong   

  1. School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, Guangdong 518000, China
  • Online:2021-11-15 Published:2021-11-16



  1. 哈尔滨工业大学(深圳) 计算机科学与技术学院,广东 深圳 518000


In the process of neural network model training, some filters degenerate into invalid filters and lose their function in the process of neural network inference. In order to solve this problem, a new learning paradigm which using the model itself can efficiently re-activate the invalid filters and improve the capability of model is proposed. Firstly, a Convolutional Neural Network(CNN) is initialized and well-trained using general methods. Then, the importance of convolution filters is measured using two methods, including L1-norm and filter correlation. Finally, weights of invalid filters are rewound to their values earlier in training, and the whole network is re-trained. Extensive experiments on classification tasks using CIFAR-10 and CIFAR-100 datasets demonstrate the effectiveness of this learning paradigm. Training anneal is applicable both on residual and lightweight networks, and invalid filters are re-activated effectively. Compared with the previous methods to improve CNN model, training anneal achieves the best effect at low cost. The accuracy of image classification is improved by 0.93% on average.

Key words: Convolutional Neural Network(CNN), image classification, filter replacement, filter efficiency



关键词: 卷积神经网络(CNN), 图像分类, 卷积核替换, 卷积核有效性