Lie Group Feature Representation Method Applied to Head Behavior Recognition in Classroom Environment

doi:10.3778/j.issn.1002-8331.2009-0510

Abstract

Abstract: Head behavior is an important part of individual behavior, especially for students’ behavior in the classroom environment. There are some well-known limitations in using RGB images for head behavior recognition such as background clutter and illumination changes. Depth maps can handle these problems well with the included depth information. Aiming at the problem of head behavior recognition in the classroom environment, inspired by Lie group theory, this paper proposes a new feature representation model based on Lie group from depth map, and designs a head behavior recognition method based on this feature representation. Key points and segments of face are constructed from 3D coordinates in depth map. The Lie group feature is extracted from the relationship between the key segments of adjacent frames, which can represent the spatial and temporal relationship of head behavior at the same time. Support vector machine（SVM） is applied to identify head behavior in this method. This work verifies the effectiveness of the proposed method on public data sets. Then the real behavior data in the classroom environment is obtained through Kinect. The experimental results show that the Lie group feature representation method can effectively help the recognition of head behavior in the classroom environment. The method can provide help for the recognition of student behavior in the classroom environment.

Key words: head behavior recognition, Lie group feature, classroom environment

摘要： 头部行为是个体行为的重要组成部分，在课堂环境下对于学生的行为来说更是如此。使用传统的RGB视频图像进行头部行为识别有着许多限制，例如背景的干扰和光线的变化等，而深度图像可以通过包含的深度信息很好地处理这些问题。针对课堂环境下的头部行为识别问题，受到李群理论的启发，提出了一种从深度图像中提取李群特征表示的模型，并且使用该李群特征完成了头部行为识别任务。从深度图像中获取脸部的关键点及关键段信息，通过计算相邻帧之间关键段的旋转及位移获得能够同时表示时间空间信息的李群特征表示，使用支持向量机来完成头部行为的分类识别。在公开数据集上验证了方法的有效性，然后通过Kinect获取制作了课堂环境下的真实行为数据，实验结果表明李群特征表示方法能够有效帮助课堂环境下头部行为的识别，对课堂环境下的学生行为识别提供了帮助。

关键词: 头部行为识别, 李群特征, 课堂环境

XIE Dong, MENG Fanrong, HE Hengtao, YAN Qiuyan. Lie Group Feature Representation Method Applied to Head Behavior Recognition in Classroom Environment[J]. Computer Engineering and Applications, 2022, 58(6): 164-169.

谢冬, 孟凡荣, 贺恒桃, 闫秋艳. 课堂环境下用于头部行为识别的李群特征表示[J]. 计算机工程与应用, 2022, 58(6): 164-169.

References

[1] NI C X.Review of computer vision research[J].Electronic World，2018（1）：91.
[2] LI K F，FAN H T.Autobiography of LiKaiFu：the world is different because of you[J].China Market，2010（12）：15.
[3] GU X L.Study on pattern recognition of head Movement based on muscle tone signal and its application[D].Shanghai：East China University of Science and Technology，2018.
[4] HU X Z，JMOUR T M.Driver status monitoring system based on Kinect sensor[J].Value Engineering，2014（20）：203-204.
[5] CHUI K T，ALHALABI W，LIU R W.Head motion coefficientbased algorithm for distracted driving detection[J].Data Technologies and Applications，2019，53（2）：171-188.
[6] FAN Z J，XU W，LIU F F，et al.Kinect based dynamic recognition method for Learners’ head posture[J].Computer and Digital Engineering，2017，45（2）：360-366.
[7] HU Z F.Identification of unsafe operation of bus drivers based on in-depth information[D].Beijing：North China University of Technology，2016.
[8] OHTSUKA H，KATO T，SHIBASATO K，et al.Non-contact head gesture maneuvering system for electric wheelchair using a depth sensor[C]//2015 9th International Conference on Sensing Technology（ICST），New Zealand，8-10 Dec 2015：98-103.
[9] OHTSUKA H，KASHIMOTO T.Simple non-contact detection method of head gesture using a depth sensor[C]//Proceedings of the 2014 IEEE/SICE International Symposium on System Integration，At Chuo University，Tokyo，2014：292-297.
[10] JINDAI M，NAKAMURA K，WATANABE T.A nodding detection system based on the active appearance model[J].Journal of Advanced Mechanical Design，Systems，and Manufacturing，2016，10（2）：21.
[11] PATWARDHAN A.Three-dimensional，kinematic，human behavioral pattern-based features for multimodal emotion recognition[J].Multimodal Technologies and Interaction，2017，1（3）：19.
[12] YANG S F，SUN T，HUANG T，et al.Group representation of line transformation rigid body motion matrix[J].Journal of Mechanical Engineering，2015，51（13）：81-85.
[13] YANG M D，LI F C，ZHANG L.Ten year research progress of machine learning in Li Qun[J].Chinese Journal of Computers，2015，38（7）：1337-1356.
[14] LIU K N，FENG X Y，SHAO C.A manifold learning dimension reduction algorithm for image classification[J].Computer Applications and Software，2019，36：8.
[15] ZHANG Y.Dimension reduction algorithm based on manifold learning and its application in face recognition[D].Dalian：Liaoning Normal University，2019.
[16] LI B.Research on manifold learning based feature extraction method and its application[D].Hefei：University of Science and Technology of China，2018.
[17] OERTEL C，MORA K A F，SHEIKHI S，et al.Who will get the grant?：a multimodal corpus for the analysis of conversational behaviours in group interviews[C]//Proceedings of the 2014 Workshop on Understanding and Modeling Multiparty，Multimodal Interactions，Istanbul Turkey，November 2014.New York：ICMI，2014：27-32.
[18] ZHU Y，CHEN W，GUO G.Fusing spatiotemporal features and joints for 3d action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2013：486-491.
[19] ARAN O，ARI I，GUVENSAN A，et al.A database of nonmanual signs in turkish sign language[C]//Signal Processing & Communications Applications，Eskisehir，Turkey，04 September 2007：1-4.
[20] AKAK N H，BüLENT S.Robust classification of face and head gestures in video[J].Image & Vision Computing，2011，29（7）：470-483.
[21] OERTEL C，GUSTAFSON J，BLACK A W.Towards building an attentive artificial listener：on the perception of attentiveness in feedback utterances[C]//ACM International Conference on Multimodal Interaction，2016：2915-2919.