Weakly Supervised Fine-Grained Image Classification Based on Xception Network

doi:10.3778/j.issn.1002-8331.2008-0402

Abstract

Abstract: With the rapid development of deep learning, the classification of images research in the field of computer vision is not only limited to recognizing the categories of objects, but also needs more detailed classification based on the traditional image classification task. Based on the existing fine-grained image classification algorithm and model analysis, a model based on Xception model and WSDAN（weakly supervised data augmentation network） weak supervision data augment method of combination of deep learning network is applied to fine-grained image classification task. The method takes Xception network as the backbone network and feature extraction network, uses the improved WSDAN model for data augment, and feeds the augmented image back to the network as the input image to enhance the generalization ability of the network. Experiments on the commonly used fine-grained image data sets and NABirds data set show that the classification accuracy rate is 89.28%, 91.18%, 94.47%, 93.04% and 88.4%, respectively. The experimental results show that this method achieves better classification results compared with WSDAN（Pytorch） model and other mainstream fine-grained classification algorithms.

Key words: fine-grained image classification, data augment, deep learning, weak supervision, Xception network

摘要： 随着深度学习的快速发展，计算机视觉领域对图像的分类研究不仅仅局限于识别出物体的类别，更需要在传统图像分类任务的基础上进行更细致的类别划分。通过对现有细粒度图像分类算法和模型的分析研究，提出一种基于Xception模型与WSDAN（weakly supervised data augmentation network）弱监督数据增强的方法相结合的深度学习网络应用于细粒度图像分类任务。该方法以Xception网络作为骨干网络和特征提取网络、利用改进的WSDAN模型进行数据增强，并把增强后的图像反馈回网络作为输入图像来增强网络的泛化能力。在常用的细粒度图像数据集和NABirds数据集上进行实验验证，得到的分类正确率分别为89.28%、91.18%、94.47%、93.04%和88.4%。实验结果表明，与WSDAN（Pytorch）模型及其他多个主流细粒度分类算法相比，该方法取得了更好的分类结果。

关键词: 细粒度图像分类, 数据增强, 深度学习, 弱监督, Xception网络

DING Wenqian, YU Pengfei, LI Haiyan, LU Xinwei. Weakly Supervised Fine-Grained Image Classification Based on Xception Network[J]. Computer Engineering and Applications, 2022, 58(2): 235-243.

丁文谦, 余鹏飞, 李海燕, 陆鑫伟. 基于Xception网络的弱监督细粒度图像分类[J]. 计算机工程与应用, 2022, 58(2): 235-243.

References

[1] 罗建豪，吴建鑫.基于深度卷积特征的细粒度图像分类研究综述[J].自动化学报，2017，43（8）：1306-1318.
LUO J H，WU J X.A survey on the classification of fine-grained images based on deep convolution features[J].Acta Automatica Sinica，2017，43（8）：1306-1318.
[2] 郑远攀，李广阳，李晔.深度学习在图像识别中的应用研究综述[J].计算机工程与应用，2019，55（12）：20-36.
ZHENG Y P，LI G Y，LI Y.Survey of application of deep learning in image recognition[J].Computer Engineering and Applications，2019，55（12）：20-36.
[3] 张潜，桑军，吴伟群，等.基于Xception的细粒度图像分类[J].重庆大学学报，2018，41（5）：85-91.
ZHANG Q，SANG J，WU W Q，et al.Fine-grained image classification based on Xception[J].Journal of Chongqing University，2018，41（5）：85-91.
[4] WEI X S，XIE C W，WU J X，et al.Mask-CNN：localizing parts and selecting descriptors for fine-grained bird species categorization[J].Pattern Recogn，2018，76：704-714.
[5] LAM M，MAHASSENI B，TODOROVIC S.Fine-grained recognition as HSnet search for informative image parts[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，Honolulu，HI，USA，2017：6497-6506.
[6] LIN T Y，ROYCHOWDHURY A，MAJI S.Bilinear CNN models for fine-grained visual recognition[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），Santiago，Chile，2015：1449-1457.
[7] FU J，ZHENG H，TAO M.Look closer to see better：recurrent attention convolutional neural network for fine-grained image recognition[C]//IEEE Conference on Computer Vision & Pattern Recognition，Honolulu，HI，USA，2017：4438-4446.
[8] 黄伟锋，张甜，常东良，等.基于多视角融合的细粒度图像分类方法[J].信号处理，2020，36（9）：1607-1614.
HUANG W F，ZHANG T，CHANG D L，et al.Multi-view comprehensive based fine-grained image classification[J].Journal of Signal Processing，2020，36（9）：1607-1614.
[9] HU T，QI H，HUANG Q.See better before looking closer：weakly supervised data augmentation network for fine-grained visual classification[DB/OL].（2019-01-26）[2019-05-23].https：//arxiv.org/abs/1901.09891.
[10] DEVRIES T，TAYLOR G W.Improved regularization of convolutional neural networks with cutout[DB/OL].（2017-08-15）[2017-11-29].https：//arxiv.org/abs/1708.04552.
[11] CHOLLET F.Xception：deep learning with depthwise separable convolutions[DB/OL].（2016-10-07）[2017-04-04].https：//arxiv.org/abs/1610.02357.
[12] WAH C，BRANSON S，WELINDER P，et al.The Caltech-UCSD birds-200-2011 dataset[Z].California Institute of Technology，2011：38-42.
[13] KHOSLA A，JAYADEVAPRAKASH N，YAO B，et al.Novel dataset for fine-grained image categorization[C]//First Workshop on Fine-Grained Visual Categorization（FGVC），IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2011.
[14] KRAUSE J，STARK M，DENG J，et al.3D object representations for fine-grained categorization[C]//2013 IEEE International Conference on Computer Vision Workshops，Sydney，2013：554-561.
[15] MAJI S，RAHTU E，KANNALA J，et al.Fine-grained visual classification of aircraft[DB/OL].[2013-01-21].https：//arxiv.org/abs/1306.5151.
[16] HORN G V，BRANSON S，FARRELL R，et al.Building a bird recognition app and large scale dataset with citizen scientists：the fine print in fine-grained dataset collection[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Boston，MA，2015：595-604.
[17] WANG Y，MORARIU V I，DAVIS L S.Learning a discriminative filter bank within a CNN for fine-grained recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：4148-4157.
[18] YANG Z，LUO T，WANG D，et al.Learning to navigate for fine-grained classification[C]//European Conference on Computer Vision，2018：438-454.
[19] SZEGEDY C，VANHOUCKE V，IOFFE S，et al.Rethinking the inception architecture for computer vision[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Las Vegas，NV，2016：2818-2826.
[20] LI P，XIE J，WANG Q，et al.Towards faster training of global covariance pooling networks by iterative matrix square root normalization[C]//IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2018：947-955.
[21] SUN M，YUAN Y，ZHOU F，et al.Multi-attention multiclass constraint for fine-grained image recognition[C]//European Conference on Computer Vision，2018：834-850.
[22] CHI Y Z，SAMY B，MORITZ H，et al.Understanding deep learning requires rethinking generalization[DB/OL].（2016-11-10）[2017-03-26].https：//arxiv.org/abs/1611.03530.
[23] GUO P，FARRELL R.Aligned to the object，not to the image：a unified pose-aligned representation for fine-grained recognition[DB/OL].（2018-01-27）[2018-09-11].https：//arxiv.org/abs/1801.09057.
[24] ZHUANG P，WANG Y，QIAO Y.Learning attentive pairwise interaction for fine-grained classification[DB/OL].[2020-02-24].https：//arxiv.org/abs/2002.10191.
[25] LUO W，YANG X，MO X，et al.Cross-X learning for fine-grained visual categorization[DB/OL].[2019-09-10].https：//arxiv.org/abs/1909.04412.
[26] 陈幻杰，王琦琦，杨国威，等.多尺度卷积特征融合的SSD目标检测算法[J].计算机科学与探索，2019，13（6）：1049-1061.
CHEN H J，WANG Q Q，YANG G W，et al.SSD object detetection algorithm with multi-scale convolution feature fusion[J].Journal of Frontiers of Computer Science and Technolody，2019，13（6）：1049-1061.