Research and Implementation of Semantic Constraint Verification Algorithm in Object Detection

doi:10.3778/j.issn.1002-8331.2009-0509

Abstract

Abstract: The aim of basic object detection consists in identifying objects in an image, as well as classifying and localizing them. However, in many application scenarios, object detection is often semantically constrained. These constraints typically include the quantitative constraint to a single category object and spatial position constraint to multiple objects. For an instance, in a video surveillance system for manufacture safety, the object detection is designed to not only identify and label safety protective equipment, but also verify the appropriate usage of it under specifications. This paper proposes a checking algorithm with semantic constraint for object detection. Firstly, a semantic constraint model is defined, and then objects in an image are detected with semantic information. Finally, the consistency of the preliminary detecting result and the semantic constraint is verified. The validity of thissemantic constraint verification algorithm is evaluated by an application instance on protective equipment detection in electricity construction and on-site safety surveillance video.

Key words: object detection, semantic constraint, template matching

摘要： 基本的目标检测任务是在图像中识别目标，并标注目标的类别和位置信息。但是，很多应用中的目标检测任务常常带有语义约束，典型的包括单类别目标的数量约束和多个目标之间的空间位置约束。如在基于视频的生产安全监控系统中，目标检测不仅要识别和标定安全防护装备，还要检测这些安全防护装备是否被规范穿戴。提出了一种目标检测中语义约束检查算法，定义一种语义约束的模型，然后对图像进行带有语义信息的目标检测，最终对目标检测结果与语义约束进行一致性判定。以电力施工防护装备检查的实际需求和现场安监视频为例，验证了所提出的目标检测中语义约束检查算法的有效性。

关键词: 目标检测, 语义约束, 模板匹配

YANG Jiayun, YAO Yinuo, YU Kun, LIU Xiumei, YU Minghe, ZHAO Zhibin. Research and Implementation of Semantic Constraint Verification Algorithm in Object Detection[J]. Computer Engineering and Applications, 2022, 58(7): 237-242.

杨佳云, 么一诺, 于鲲, 柳秀梅, 于明鹤, 赵志滨. 目标检测中语义约束检查算法的研究与实现[J]. 计算机工程与应用, 2022, 58(7): 237-242.

References

[1] HU H，GU J，ZHANG Z，et al.Relation networks for object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2018：3588-3597.
[2] ZOU Z X，SHI Z W，GUO Y H，et al.Object detection in 20 years：A survey[J].arXiv：1905.05055，2019.
[3] GIRSHICK R，DONAHUE J，DARRELL T，et al.Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2014：580-587.
[4] GIRSHICK R.Fast R-CNN[C]//Proceedings of the IEEE International Conference on Computer Vision（ICCV），2015：1440-1448.
[5] REN S Q，HE K M，GIRSHICK R，et al.Faster R-CNN：Towards real-time object detection with region proposal networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2015，39（6）：1137-1149.
[6] LIU W，ANGUELOV D，ERHAN D，et al.SSD：Single shot multibox detector[C]//Proceedings of European Conference on Computer Vision（ECCV），2016：21-37.
[7] REDMON J，DIVVALA S，GIRSHICK R，et al.You only look once：Unified，real-time object detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2016：779-788.
[8] REDMON J，FARHADI A.YOLO9000：Better，faster，stronger[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition（CVPR），2017：6517-6525.
[9] REDMON J，FARHADI A.YOLOv3：An incremental improvement[J].arXiv：1804.02767，2018.
[10] MATOUSEK J.On the L2-discrepancy for anchored boxes[J].Journal of Complexity，1998，14（4）：527-556.
[11] YAMAGUCHI N.Combining pairwise coupling classifiers using individual logistic regressions[C]//Proceedings of International Conference on Neural Information Processing，2006：11-20.
[12] TORRALBA A.Contextual priming for object detection[J].International Journal of Computer Vision，2003，53（2）：169-191.
[13] CARBONETTO P，DE FREITAS N，BARNARD K.A statistical model for general contextual object recognition[C]//Proceedings of European Conference on Computer Vision，2004：350-362.
[14] 陈静.视觉目标的语境关联约束及时空特性分析[D].合肥：合肥工业大学，2016.
CHEN J.Contextual constraints and spatiotemporal characteristics of visual objects[D].Hefei：Hefei Polytechnic University，2016.
[15] 戴进.基于语义的视觉定位算法研究[D].哈尔滨：哈尔滨工业大学，2019.
DAI J.Research on visual location algorithm based on semantics[D].Harbin：Harbin Institute of Technology，2019.
[16] 徐守坤，倪楚涵，吉晨晨，等.一种基于安全帽佩戴检测的图像描述方法研究[J].小型微型计算机系统，2020，41（4）：812-819.
XU S K，NI C H，JI C C，et al.Research on image description method based on helmet wearing detection[J].Journal of Chinese Computer Systems，2020，41（4）：812-819.