Computer Engineering and Applications ›› 2024, Vol. 60 ›› Issue (10): 188-197.DOI: 10.3778/j.issn.1002-8331.2212-0371

• Graphics and Image Processing • Previous Articles     Next Articles

Detection and Tracking Method for Multiple Moving Objects in Panoramic Video

LIU Huitong, WANG Xi, LIU Wei, YANG Peng   

  1. Hubei Key Lab of Smart Internet, School of Electronic Information and Communications, Huazhong University of Science and Technology, Wuhan 430074, China
  • Online:2024-05-15 Published:2024-05-15

全景视频中多运动对象检测与跟踪方法

刘慧彤,王希,刘威,杨鹏   

  1. 华中科技大学 智能互联网湖北省重点实验室 电子信息与通信学院,武汉 430074

Abstract: The general solution of detection and tracking technology in panoramic videos is to detect and track moving objects after converting 360° video into two-dimensional video by projection. However, the fixed spherical projection will lead to discontinuous projection boundaries in panoramic videos. When a moving object is at a projection boundary, conventional spherical projection methods may divide the object into multiple parts, resulting in the missed or false detection. A multi-motion object detection and tracking method in panoramic video combining equi-rectangular projection (ERP) and cube map projection (CMP) is proposed. It uses suitable CMP stitching frames to reduce the polar distortion and compensate for the discontinuity of projection boundaries of ERP frames, solving the detection of objects at projection boundaries or polar distortion. Then, multi-motion object tracking is solved by fusing multiple features of perceptual hashing, spherical centroid and motion direction. Experimental results show that the proposed scheme can effectively alleviate the false detection caused by discontinuous projection boundaries and correctly track moving objects with occlusion, frame boundary shuttle or polar distortion.

Key words: panoramic video, multi-motion object detection and tracking, equi-rectangular projection (ERP), cube map projection (CMP)

摘要: 全景视频多运动对象检测与跟踪技术的通用解决方案是通过投影变换将360°视频转为二维视频后进行检测和跟踪。然而固定球面投影变换会导致全景视频中存在空间上不连续的投影边界,当运动对象处于投影边界时,常规球面投影方法可能将其切分为多个部分,从而导致该运动对象的漏检或误检。针对该问题,提出了融合等距柱状形投影(equi-rectangular projection,ERP)和立方体投影(cube map projection,CMP)的全景视频多运动对象检测与跟踪方法,利用合适的CMP拼接帧减少ERP两极区域失真并弥补其边界不连续性,解决了处于投影边界和两极扭曲的对象检测问题。进一步,通过感知哈希、球面质心和运动方向等多重特征融合来解决全景视频多运动对象跟踪问题。实验结果表明,提出的方案可以有效减少投影边界不连续导致的误检,且能有效应对遮挡、穿梭帧边界和两极扭曲的对象跟踪问题。

关键词: 全景视频, 多运动对象检测与跟踪, 等距柱状形投影(ERP), 立方体投影(CMP)