结合双金字塔特征融合与级联定位的车牌检测

doi:10.3778/j.issn.1002-8331.2108-0439

摘要/Abstract

摘要： 为了解决复杂环境中不同因素干扰车牌检测精确度的问题，提出了一种基于双金字塔特征融合的复杂环境下车牌检测算法。通过采用Mish激活函数的残差网络（ResNet101-M）对输入图像进行初级特征提取；在传统特征金字塔网络（feature pyramid network，FPN）的基础上，提出了一种改进的双金字塔特征融合网络（siamese feature pyramid network，SFPN）。被提取的初级特征被送入该网络进行多层特征融合。融合后的特征被送入基于形状先验的锚点设置网络来确定感兴趣区域。将所生成的感兴趣区域送入级联定位网络从而得到准确的车牌检测结果。实验结果表明，该算法在AOLP与CCPD车牌数据集上均能够有效提升检测性能。

关键词: 车牌检测, 深度学习, 双金字塔特征融合, 级联定位

Abstract: In order to solve the problem of license plate detection in complex environment, a novel license plate detection algorithm using siamese feature pyramid and cascaded positioning is proposed. Firstly, the original features of the images are extracted by the residual network with the Mish activation function. At the same time, a siamese feature pyramid network（SFPN） is proposed in this paper. The extracted features are sent to the siamese feature pyramid network for multi-level feature fusion. Then, a region proposal network based on the prior of shape is used to generate the regions of interest. Finally, the regions of interest are fed to the cascaded positioning network to obtain the license plate location. The experimental results show that the proposed algorithm effectively improves the performance of license plate detection on both AOLP and CCPD datasets.

Key words: license plate detection, deep learning, siamese feature pyramid, cascaded positioning

张俊青, 熊玉洁, 孙宪坤, 高永彬. 结合双金字塔特征融合与级联定位的车牌检测[J]. 计算机工程与应用, 2023, 59(2): 240-252.

ZHANG Junqing, XIONG Yujie, SUN Xiankun, GAO Yongbin. License Plate Detection Using Siamese Feature Pyramid and Cascaded Positioning[J]. Computer Engineering and Applications, 2023, 59(2): 240-252.

参考文献

[1] ANAGNOSTOPOULOS C N E，ANAGNOSTOPOULOS I E，PSOROULAS I D，et al.License plate recognition from still images and video sequences：a survey[J].IEEE Transactions on Intelligent Transportation Systems，2008，9（3）：377-391.
[2] COETZEE C，BOTHA C，WEBER D.PC based number plate recognition system[C]//Proceedings of the IEEE International Symposium on Industrial Electronics，1998：605-610.
[3] PARISI R，CLAUDIO E D D，LUCARELLI G，et al.Car plate recognition by neural networks and image processing[C]//Proceedings of the IEEE International Symposium on Circuits and Systems，1998：195-198.
[4] BARROSO J，DAGLESS E L，RAFAEL A，et al.Number plate reading using computer vision[C]//Proceedings of the IEEE International Symposium on Industrial Electronics，1997：761-766.
[5] ZHENG D N，ZHAO Y N，WANG J X.An efficient method of license plate location[J].Pattern Recognition Letters，2005，26（15）：2431-2438.
[6] ZHOU W G，LI H Q，LU Y J，et al.Principal visual word discovery for automatic license plate detection[J].IEEE Transactions on Image Processing，2012，21（9）：4269-4279.
[7] ABOLGHASEMI V，AHMADYFARD A.An edge-based color-aided method for license plate detection[J].Image and Vision Computing，2009，27（8）：1134-1142.
[8] YU S，LI B，ZHANG Q，et al.A novel license plate location method based on wavelet transform and EMD analysis[J].Pattern Recognition，2015，48（1）：114-125.
[9] AL-GHAILI A M，MASHOHOR C，RAMLI A R，et al.Vertical edge-based car license plate detection method[J].IEEE Transactions on Vehicular Technology，2013，62（1）：26-38.
[10] CHO B K，RYU S H，SHIN D R，et al.License plate extraction method for identification of vehicle at a railway level crossing[J].International Journal of Automotive Technology，2011，12（2）：281-289.
[11] AZAD R，DAVAMI F，AZAD B.A novel and robust method for automatic license plate recognition system based on pattern recognition[J].Advances in Computer Science，2013，2（3）：64-70.
[12] DEB K，KHAN I，SAHA A，et al.An efficient method of vehicle license plate recognition based on sliding concentric windows and artificial neural network[J].Procedia Technology，2012，4：812-819.
[13] 魏亭，邱实，李晨，等.计算机多尺度辅助定位车牌算法[J].电子学报，2018，46（9）：2188-2193.
WEI T，QIU S，LI C，et al.License plate detection algorithm based on computer multi scale assist[J].Acta Electronica Sinica，2018，46（9）：2188-2193.
[14] 葛艳，陈晨.基于文字定位和颜色检测的多车牌定位方法研究[J].计算机与数字工程，2018，46（3）：575-579.
GE Y，CHEN C.Research on multi-license plate location based on multi-algorithms[J].Computer & Digital Engineering，2018，46（3）：575-579.
[15] REDMON J，DIVVALA S，et al.You only look once：unified，real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：779-788.
[16] LIU W，ANGUELOV D，ERHAN D，et al.SSD：single shot multibox detector[C]//Proceedings of European Conference on Computer Vision.Heidelberg：Springer，2016：21-37.
[17] XIE L L，AHMAD T.A new CNN-based method for multi-directional car license plate detection[J].IEEE Transactions on Intelligent Transportation Systems，2018，19（2）：507-517.
[18] 刘华春，侯向宁.采用视觉显著性和卷积网络的车牌定位算法[J].计算机技术与发展，2019，29（6）：74-78.
LIU H C，HOU X N.A license plate location algorithm using visual saliency and convolutional network[J].Computer Technology and Development，2019，29（6）：74-78.
[19] MOLINA-MORENO M，GONZALEZ-DIAZ I，DIAZDEMARIA F.Efficient scale-adaptive license plate detection system[J].IEEE Transactions on Intelligent Transportation Systems，2019，20（6）：2109-2121.
[20] REN S Q，HE K M，GIRSHICK R，et al.Faster R-CNN：towards real-time object detection with region proposal networks[C]//Proceedings of Neural Information Processing System.Cambridge：MIT Press，2015：91-99.
[21] 李祥鹏，闵卫东，韩清，等.基于深度学习的车牌检测和识别方法[J].计算机辅助设计与图形学学报，2019，31（6）：979-987.
LI X P，MIN W D，HAN Q，et al.License plate location and recognition based on deep learning[J].Journal of Computer-Aided Design & Computer Graphics，2019，31（6）：979-987.
[22] HE K M，GKIOXARI G，DOLLAR P，et al.Mask R-CNN[J].IEEE Transactions on Pattern Analysis & Machine Intelligence，2018，42（2）：386-397.
[23] 赵伟，鞠美玉，李子双.复杂光线下的车牌图像增强算法[J].计算机辅助设计与图形学学报，2016，28（6）：976-985.
ZHAO W，JU M Y，LI Z S.A license plate image enhancement algorithm under complicated illumination[J].Journal of Computer-Aided Design & Computer Graphics，2016，28（6）：976-985.
[24] HE K M，ZHANG X Y，REN S Q，et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2016：770-778.
[25] MISRA D.Mish：a self regularized non-monotonic neu-ral activation function[J].arXiv：1908.08681，2019.
[26] LIN T Y，DOLLAR P，GIRSHICK R，et al.Feature pyramid networks for object detection[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision，2017：2117-2125.
[27] HSU G S，CHEN J C，CHUNG Y Z.Application-oriented license plate recognition[J].IEEE Transactions on Vehicular Technology，2013，62（2）：552-561.
[28] XU Z B，YANG W，MENG A J，et al.Towards end-to-end license plate detection and recognition：a large dataset and baseline[C]//Proceedings of European Conference on Computer Vision.Heidelberg：Springer，2018：261-277.
[29] LI H，WANG P.Toward end-to-end car license plate detection and recognition with deep neural networks[J].IEEE Transactions on Intelligent Transportation Systems，2019，20（3）：1126-1136.
[30] LI H，SHEN C H.Reading car license plates using deep convolutional neural networks and LSTMs[J].arXiv：1601.
05610，2016.
[31] SELMI Z，ALIMI A M.Deep learning system for automatic license plate detection and recognition[C]//Proceedings of the 14th IAPR International Conference on Document Analysis and Recognition，2017：1132-1138.

[32] SELMI Z，HALIMA M B，PAL U，et al.DELP-DAR system for license plate detection and recognition[J].Pattern Recognition Letters，2020，129：213-223.

[33] REDMON J，FARHADI A.YOLOv3：an incremental improvement[J].arXiv：1804.02767，2018.