Computer Engineering and Applications ›› 2023, Vol. 59 ›› Issue (19): 285-293.DOI: 10.3778/j.issn.1002-8331.2211-0367

• Engineering and Applications • Previous Articles     Next Articles

Gesture Interaction and Linkage System of Flat-Panel Display and Integral Imaging for Bones

WEI Jian, GUO Xiaoyuan, JI Fenglei, GUO Shaoyuan, HU Xiaohan, JIA Yong, WANG Shigang   

  1. 1.College of Communication Engineering, Jilin University, Changchun 130012, China
    2.The First Hospital, Jilin University, Changchun 130061, China
    3.College of Nursing, Jilin University, Changchun 130021, China
  • Online:2023-10-01 Published:2023-10-01

骨骼平面显示与集成成像手势交互联动系统

韦健,郭校源,籍风磊,郭绍源,胡笑含,贾勇,王世刚   

  1. 1.吉林大学 通信工程学院,长春 130012
    2.吉林大学 第一医院,长春 130061
    3.吉林大学 护理学院,长春 130021

Abstract: Flat-panel display interacted with the mouse and keyboard has difficulty in presenting multi-view spatial information of the human bones.Moreover, it has problems of heavily taking place and bacterial infections.Integral imaging is widely claimed to be a naked-eye three-dimensional display with the highest potential of development.A linkage system of flat-panel display and integral imaging supporting gesture interaction is proposed.The real-time performance of Optix and Leap Motion, respectively in ray casting and accurate gesture recognition, is combined.The mathematical relationship between the pose of virtual camera array and the gesture data is considered.By immediate feedback of gesture information, parameter updating of virtual cameras, and efficient rendering of content shown, synchronous adjusting the viewpoint and scale of the bone’s two-dimensional view and naked-eye three-dimensional image by viewer’s gestures is allowed.The results indicate that, the framerate of this system can reach 30 frames per second on single GTX 1080Ti GPU, when the two- and three-dimensional(26 view points) images both have scree resolution of 2?160×1?280.The effective and fluent telekinesis interaction as well as mixture and linkage display help to improve of viewing experience, sensory impact, and practical worth three-dimensional bone visualization.

Key words: integral imaging, gesture interaction, double-screen linkage, three-dimensional medical imaging

摘要: 基于鼠键控制的医用平面显示不易同时呈现骨骼的多视角空间信息,还存在占用空间和细菌感染的问题;集成成像被普遍认为是发展潜力最大的裸眼三维显示技术。提出一种支持手势交互的平面显示与集成成像联动系统,挖掘Optix和Leap Motion分别在光线投射和精准手势识别方面的实时性,根据虚拟相机阵列的位姿与手势数据之间的数学关系,通过即时的手势指令检测、虚拟相机位姿更迭及显示内容快速渲染,允许观看者利用手势同步调整骨骼二维视图和裸眼三维影像的视角和尺度。实验结果表明,该系统在二维与立体(26视点)画面的屏幕分辨率皆为2?160×1?280的情况下,在单块GTX 1080Ti GPU上的帧率达到36帧/s,有效和流畅的隔空交互、混合联动显示有助于提升骨骼三维可视化的观看体验、感官冲击和实际应用价值。

关键词: 集成成像, 手势交互, 双屏联动, 三维医学成像