Lightweight Multi-scale Attention Fusion Algorithm for License Plate Detection

doi:10.3778/j.issn.1002-8331.2012-0022

Abstract

Abstract:

License plate recognition technology plays an important role in traffic management, among which license plate detection has a significant impact on the subsequent recognition performance. The existing license plate detection system is easy to be interfered by external environment and has poor detection performance in natural scenes. In this paper, a license plate detection network model based on multi-scale attention fusion is proposed. The pyramid network feature map and CBAM（Convolutional Block Attention Module） attention structure are used to improve the detection accuracy of small targets. At the same time, this method can not only accurately detect and locate the license plate under natural scenes, but also accurately locate the four corners of the license plate, which is beneficial to the subsequent application of license plate recognition. In the experiment, the CCPD data set is amplified by data enhancement method, which effectively alleviates the influence of complex environment changes on license plate detection and enhances the robustness of the model. By training and testing the model, the average accuracy rate of 98.05% and recall rate of 98.71% are obtained, which are better than other license plate detection methods. Moreover, the frame rate reaches 64 frame/s and the real-time performance is high.

Key words: deep learning, target detection, license plate detection, attention mechanism

摘要：

车牌识别技术在交通管理中发挥着重要作用，其中车牌检测环节对后续识别性能有重大影响。现有的车牌检测系统容易受到外部环境的干扰，在自然场景下的检测性能差。提出一种基于多尺度注意力融合的车牌检测网络模型，利用金字塔网络特征图和CBAM（Convolutional Block Attention Module）注意力结构，提高小目标的检测精度。同时该方法不仅能够准确地检测定位出自然场景下的车牌，还能精确地定位出车牌的4个角点，有利于后续的车牌识别应用。实验中采用数据增强方法对CCPD数据集进行扩增，有效缓解了复杂环境变化对车牌检测造成的影响，增强了模型鲁棒性。通过对模型进行训练和测试，获得了98.05%的平均精确率和98.71%的召回率，优于其他车牌检测方法，并且帧率达到64?frame/s，实时性高。

关键词: 深度学习, 目标检测, 车牌检测, 注意力机制

ZHANG Shihao, YANG Xiujun, WU Linhuang, CHEN Pingping. Lightweight Multi-scale Attention Fusion Algorithm for License Plate Detection[J]. Computer Engineering and Applications, 2021, 57(22): 208-214.

张世豪，杨绣郡，吴林煌，陈平平. 轻量型多尺度注意力融合的车牌检测算法[J]. 计算机工程与应用, 2021, 57(22): 208-214.

References

[1] WANG W H，TU J Y.Research on license plate recognition algorithms based on deep learning in complex environment[J].IEEE Access，2020，8：91661-91675.
[2] PENG X，WEN L，BAI D，et al.Reformative vehicle license plate recognition algorithm based on deep learning[C]//International Conference on Cognitive Systems and Signal Processing.Singapore：Springer，2018：243-255.
[3] HOWARD A G，ZHU M，CHEN B，et al.MobileNets：efficient convolutional neural networks for mobile vision applications[J].arXiv：1704.04861，2017.
[4] NAJIBI M，SAMANGOUEI P，CHELLAPPA R，et al.SSH：single stage headless face detector[C]//2017 IEEE International Conference on Computer Vision，2017.
[5] YANG T，LONG X，SANGAIAH A K，et al.Deep detection network for real-life traffic sign in vehicular networks[J].Computer Networks，2018，136：95-104.
[6] YUAN Y，XIONG Z T，WANG Q.VSSA-NET：vertical spatial sequence attention network for traffic sign detection[J].IEEE Transactions on Image Processing，2019，28（7）：3423-3434.
[7] SHI X，ZHAO W，SHEN Y.Automatic license plate recognition system based on color image processing[C]//International Conference on Computational Science and Its Applications.Berlin，Heidelberg：Springer，2005：1159-1168.
[8] TIAN Y，SONG J，ZHANG X，et al.An algorithm combined with color differential models for license-plate location[J].Neurocomputing，2016，212：22-35.
[9] BAI H L，LIU C P.A hybrid license plate extraction method based on edge statistics and morphology[C]//17th International Conference on Pattern Recognition，2004.
[10] HSU G S，CHEN J C，CHUNG Y Z.Application-oriented license plate recognition[J].IEEE Transactions on Vehicular Technology，2013，62（2）：552-561.
[11] ZHOU W，LI H，LU Y，et al.Principal visual word discovery for automatic license plate detection[J].IEEE Transactions on Image Processing，2012，21（9）：4269-4279.
[12] LI B，TIAN B，LI Y，et al.Component-based license plate detection using conditional random field model[J].IEEE Transactions on Intelligent Transportation Systems，2013，14（4）：1690-1699.
[13] 陈俊，何小海，滕奇志.基于最大稳定极限区域的车牌定位[J].科学技术与工程，2015，15（31）：212-217.
CHEN J，HE X H，TENG Q Z.License plate localization based on MSER[J].Science Technology and Engineering，2015，15（31）：212-217.
[14] LI Q.A geometric framework for rectangular shape detection[J].IEEE Transactions on Image Processing，2014，23（9）：4139-4149.
[15] LI H，SHEN C.Reading car license plates using deep convolutional neural networks and LSTMs[J].Image and Vision Computing，2018，72：14-23.
[16] RAFIQUE M A，PEDRYCZ W，JEON M.Vehicle license plate detection using region-based convolutional neural networks[J].Soft Computing，2017，22（3）：1-12.
[17] IANDOLA F N，HAN S，MOSKEWICZ M W，et al.SqueezeNet：AlexNet-level accuracy with 50x fewer parameters and <0.5 MB model size[J].arXiv：1602.07360，2016.
[18] ZHANG X，ZHOU X，LIN M，et al.ShuffleNet：an extremely efficient convolutional neural network for mobile devices[C]//2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：6848-6856.
[19] LIU S，HUANG D.Receptive field block net for accurate and fast object detection[C]//15th European Conference on Computer Vision，2018：385-400.
[20] WOO S，PARK J，LEE J Y，et al.CBAM：convolutional block attention module[C]//15th European Conference on Computer Vision，2018：3-19.
[21] XU Z，YANG W，MENG A，et al.Towards end-to-end license plate detection and recognition：a large dataset and baseline[C]//15th European Conference on Computer Vision，2018：255-271.
[22] DENG J，GUO J，ZHOU Y，et al.RetinaFace：single-stage dense face localisation in the wild[J].arXiv：1905.00641，2019.
[23] LI H，WANG P，SHEN C.Toward end-to-end car license plate detection and recognition with deep neural networks[J].IEEE Transactions on Intelligent Transportation Systems，2018，20（3）：1126-1136.
[24] LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//14th European Conference on Computer Vision.Cham：Springer，2016：21-37.
[25] REN S，HE K，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[C]//Advances in Neural Information Processing Systems，2015：91-99.
[26] REDMON J，FARHADI A.YOLO9000：better，faster，stronger[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition，2017：7263-7271.
[27] WANG W，YANG J，CHEN M，et al.A light CNN for end-to-end car license plates detection and recognition[J].IEEE Access，2019，7：173875-173883.