Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (6): 134-145.DOI: 10.3778/j.issn.1002-8331.2110-0083

• Pattern Recognition and Artificial Intelligence • Previous Articles     Next Articles

Research on Multi-Path Lightweight Convolutional Neural Network

ZHAO Lixin, BAI Yu, AN Shengbiao   

  1. School of Information Science and Engineering, Hebei University of Science and Technology, Shijiazhuang 050018, China
  • Online:2023-03-15 Published:2023-03-15



  1. 河北科技大学 信息科学与工程学院,石家庄 050018

Abstract: Numerous studies have shown that network width widening can extract more multivariate features, but the number of parameters grows quadratically when the model channel width is widened, leading to high training costs and large network models. To address these problems, a multi-path module is proposed. By optimizing the calculation structure of the internal path of the module, the model can extract multiple features economically and efficiently. Specifically, compared with the traditional network width expansion method, the multi-pathway module avoids the expansion in the channel dimension and shifts the expansion method to the pathway dimension, so that the number of model parameters can be effectively reduced while ensuring the diversity of output features. Since it is easier to extract abstract semantical information by depth deepening, the multi-path module has multi-scale feature extraction capability by setting different convolution depths for each computational pathway (the feature map contains both detailed location information and gradually abstract semantic information). During the research process, as this paper tries to improve the pathway relationship within the module by invoking the attention mechanism, and observes that the attention mechanism has certain conditionality. Experimental results show that the 5.3 MB lightweight convolutional neural network composed of this multi-path module has a classification accuracy of 1.32% higher than that of Res-Net-18 with 43.4 MB on CiFar-10, and after optimization of the model (33 MB), the classification accuracy reaches 95.15% higher than that of SE-Net18 (45.1 MB) by 0.65%.

Key words: multi-path, light-weighting, width, feature distribution, attention mechanism

摘要: 大量研究表明,卷积神经网络宽度展宽可以提取更加多元的特征,但对模型通道宽度进行展宽时,引起训练参数量呈二次增长,进而导致训练成本高与网络模型庞大的问题。针对上述问题,提出多通路模块,通过优化模块内部通路的运算结构,使得模型可以经济高效地提取多元特征。具体来说,与传统网络宽度展宽方式相比,多通路模块避免在通道维度上展宽,将展宽方式转移到通路维度,保证输出特征多样性的同时有效降低了模型参数量。由于网络深度加深更容易提取抽象的语义信息,多通路模块通过对各计算通路设定不同的卷积深度,使得模块具有多尺度特征提取能力,即使得输出特征中既包含细节位置信息,也包括逐渐抽象的语义信息。在研究过程中,利用注意力机制改善模块内通路关系时,发现注意力机制存在一定条件性。实验结果表明,由多通路模块组成的5.3?MB轻量化卷积神经网络,在CiFar-10上分类准确度比43.4?MB的Res-Net-18高出1.32%。对模型优化后(33?MB),分类准确度达到了95.15%,较SE-Net18(45.1?MB)精度提升0.65%。

关键词: 多通路, 轻量化, 网络宽度, 特征分布, 注意力机制