Fine-Grained Vehicle Model Recognition Based on Mixed Class Balance Loss

doi:10.3778/j.issn.1002-8331.2205-0543

Abstract

Abstract: In order to deal with the problem that the head category is overfitting and the tail category is ignored during training due to the uneven distribution of vehicle model data, a method for fine-grained vehicle model recognition based on mixed class balance loss is proposed. A mixed class balance cross-entropy loss function is proposed by combining the Mixup data augmentation method and the class-balance loss. The balanced subset fine-tuning is used as the training strategies to further improve the recognition effect of long-tailed distribution data. The experimental results show that the accuracy on Stanford Cars, CompCars, and SYSU Cars datasets is improved by 1.07, 0.17, and 1.58?percentage points, respectively, which effectively alleviates the problems caused by the imbalanced data and improves the recognition effect of vehicle model recognition even further. The SYSU Cars is a self-built dataset, which contains 102 vehicle brands, 691 models and various lighting scenes（to be available on OpenITS soon）.

Key words: fine-grained vehicle model recognition, fine-grained recognition, mixed class balance loss, long-tailed distribution

摘要： 为了应对车型精细识别中数据分布不均衡导致训练中头部类别过拟合，而尾部类别被忽略的问题，提出了一种基于混合类别均衡损失的车型精细识别数据增强方法。结合Mixup数据增强方法和类别均衡损失，提出混合类别均衡交叉熵损失函数；通过均衡子集微调的训练策略，进一步提高了长尾分布数据的识别效果。实验结果表明，算法在Stanford Cars、CompCars、SYSU Cars数据集上的识别准确率分别比Baseline提高了1.07、0.17和1.58个百分点，有效地缓解了因车型数据不均衡带来的问题，进一步提高了车型精细识别的识别效果。其中SYSU Cars为自建数据集，由66?137张车辆正脸图片构成，包含102种品牌，691种车型以及不同的光照条件（即将在OpenITS上公开）。

关键词: 车型精细识别, 细粒度识别, 混合类别均衡损失, 长尾分布

LI Xiying, QUAN Fengwei, YE Zhihui. Fine-Grained Vehicle Model Recognition Based on Mixed Class Balance Loss[J]. Computer Engineering and Applications, 2023, 59(17): 187-194.

李熙莹, 全峰玮, 叶芝桧. 基于混合类别均衡损失的车型精细识别[J]. 计算机工程与应用, 2023, 59(17): 187-194.

References

[1] WAH C，BRANSON S，WELINDER P，et al.The Caltech-UCSD Birds-200-2011 dataset：CNS-TR-2011-001[R].2011.
[2] KHOSLA A，JAYADEVAPRAKASH N，YAO B，et al.Novel dataset for fine-grained image categorization[C]//First Workshop on Fine-Grained Visual Categorization（FGVC），2011.
[3] MAJI S，RAHTU E，KANNALA J，et al.Fine-grained visual classification of aircraft[J].arXiv：1306.5151，2013.
[4] FANG J，ZHOU Y，YU Y，et al.Fine-grained vehicle model recognition using a coarse-to-fine convolutional neural network architecture[J].IEEE Transactions on Intelligent Transportation Systems，2017，18（7）：1782-1792.
[5] DAI X，SOUTHALL B，TRINH N，et al.Efficient fine-grained classification and part localization using one compact network[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision（ICCV），Venice，Italy，October 22-29，2017：996-1004.
[6] 杨娟，曹浩宇，汪荣贵，等.基于语义DCNN特征融合的细粒度车型识别模型[J].计算机辅助设计与图形学学报，2019，31（1）：141-157.
YANG J，CAO H Y，WANG R G，et al.Fine-grained car recognition model based on semantic DCNN features fusion[J].Journal of Computer-Aided Design & Computer Graphics，2019，31（1）：141-157.
[7] LIN T，ROYCHOWDHURY A，MAJI S.Bilinear CNN models for fine-grained visual recognition[C]//2015 IEEE International Conference on Computer Vision（ICCV），Santiago，Chile，December 7-13，2015：1449-1457.
[8] HU Q，WANG H，LI T，et al.Deep CNNs with spatially weighted pooling for fine-grained car recognition[J].IEEE Transactions on Intelligent Transportation Systems，2017，18（11）：3147-3156.
[9] HU T，QI H，HUANG Q，et al.See better before looking closer：weakly supervised data augmentation network for fine-grained visual classification[J].arXiv：1901.09891，2019.
[10] MA Z，CHANG D，XIE J，et al.Fine-grained vehicle classification with channel max pooling modified CNNs[J].IEEE Transactions on Vehicular Technology，2019，68（4）：3224-3233.
[11] 宋岩贝，魏维，何冰倩.基于中层特征的细粒度的车型识别[J].计算机工程与设计，2020，41（6）：1708-1713.
SONG Y B，WEI W，HE B Q.Fine-grained vehicle type recognition based on mid-level features[J].Computer Engineering and Design，2020，41（6）：1708-1713.
[12] 李致金，张亮，武鹏，等.基于特征融合卷积神经网络的车型精细识别[J].计算机工程与设计，2020，41（1）：226-230.
LI Z J，ZHANG L，WU P，et al.Fine-grained vehicle models based on feature fusion convolutional neural network[J].Computer Engineering and Design，2020，41（1）：226-230.
[13] 刘廷建，顾乃杰，张孝慈，等.基于多尺度特征融合CNN模型的车辆精细型号识别[J].计算机工程与应用，2018，54（18）：154-160.
LIU T J，GU N J，ZHANG X C，et al.Fine-grained recognition of vehicle model using multi-scale feature fusion CNN[J].Computer Engineering and Applications，2018，54（18）：154-160.
[14] ZHANG X，ZHOU F，LIN Y，et al.Embedding label structures for fine-grained feature representation[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Las Vegas，USA，June 27-30，2016：1114-1123.
[15] EM Y，GAG F，LOU Y，et al.Incorporating intra-class variance to fine-grained visual recognition[C]//2017 IEEE International Conference on Multimedia and Expo（ICME），Hong Kong，China，July 10-14，2017：1452-1457.
[16] LI X，YU L，CHANG D，et al.Dual cross-entropy loss for small-sample fine-grained vehicle classification[J].IEEE Transactions on Vehicular Technology，2019，68（5）：4204-4212.
[17] 李哲，胡朋立，邓军勇.基于局部特征与多损失融合的车型精细识别算法[J].传感器与微系统，2021，40（4）：142-145.
LI Z，HU P L，DENG J Y.Vehicle model fine recognition algorithm based on local features and multiple losses[J].Transducer and Microsystem Technologies，2021，40（4）：142-145.
[18] KRAUSE J，STARK M，DENG J，et al.3D object representations for fine-grained categorization[C]//2013 IEEE International Conference on Computer Vision（ICCV），Sydney，Australia，December 1-8，2013：554-561.
[19] YANG L，LUO P，LOY C C，et al.A large-scale car dataset for fine-grained categorization and verification[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Boston，USA，June 7-12，2015：3973-3981.
[20] KRIZHEVSKY A，SUTSKEVER I，HINTON G.ImageNet classification with deep convolutional neural networks[C]//2012 26th Neural Information Processing Systems（NeurIPS），Lake Tahoe，USA，December 3-6，2012：1106-1114.
[21] SZEGEDY C，LIU W，JIA Y，et al.Going deeper with convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Boston，USA，June 7-12，2015：1-9.
[22] ZHANG H，CISSE M，DAUPHIN Y N，et al.Mixup：beyond empirical risk minimization[C]//2018 6th International Conference on Learning Representations（ICLR），Vancouver，Canada，April 30-May 3，2018.
[23] CUI Y，JIA M，LIN T Y，et al.Class-balanced loss based on effective number of samples[C]//2019 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Long Beach，USA，June 16-20，2019：9268-9277.