改进YOLOv8的学生课堂行为识别算法：DMS-YOLOv8

doi:10.3778/j.issn.1002-8331.2407-0132

摘要/Abstract

摘要： 针对智慧教室中存在前后排学生图像尺寸差异较大、后排小目标检测困难的问题，提出了一种改进YOLOv8的学生课堂行为识别方法：DMS-YOLOv8。结合CA注意力机制与深度卷积，提出了动态通道注意力卷积（DCAConv），能够动态调整通道权重，更灵敏地捕获关键特征；引入多尺度卷积注意力（MSCA），通过元素乘法最大化挖掘多尺度卷积特征，增强对空间细节的关注；同时，构建了多尺度上下文融合（LCD）模块，通过卷积和自注意力机制，增强多尺度特征融合。增加小目标检测层，通过较大尺寸特征图的局部特征提取，显著提高模型对后排学生行为的识别能力。与基线模型YOLOv8n相比，该方法在自制学生行为数据集上的mAP50值提高了4.6个百分点，在VOC数据集上提高了18.7个百分点。该方法在学生课堂行为识别方面表现突出，可显著提高智慧教室学生课堂行为识别的准确率。

关键词: 学生行为识别, YOLOv8, 目标检测, 动态通道注意力卷积, 多尺度上下文融合

Abstract: To address significant image size differences and small target detection challenges in smart classrooms, an improved YOLOv8 method for recognizing student behavior, DMS-YOLOv8, is proposed. Firstly, dynamic channel attention convolution (DCAConv) combines CA attention with deep convolution to dynamically adjust channel weights and capture key features. Secondly, multi-scale convolutional attention (MSCA) utilizes element-wise multiplication to enhance spatial details by maximizing multi-scale features. Additionally, a multi-scale context fusion (LCD) module is constructed to improve feature fusion using convolution and self-attention mechanisms. Finally, a small target detection layer is added to enhance the model’s ability to recognize back-row student behavior by extracting local features from larger-sized feature maps. Compared to the baseline YOLOv8n model, this method improves the mAP50 value by 4.6 percengtage points on a custom student behavior dataset and by 18.7 percengtage points on the VOC dataset, significantly increasing the accuracy of student classroom behavior recognition in smart classrooms.

Key words: student behavior recognition, YOLOv8, object detection, dynamic channel attention convolution, multi-scale context fusion

陈晨, 保文星, 陈旭, 景永俊, 李卫军. 改进YOLOv8的学生课堂行为识别算法：DMS-YOLOv8[J]. 计算机工程与应用, 2024, 60(24): 222-234.

CHEN Chen, BAO Wenxing, CHEN Xu, JING Yongjun, LI Weijun. Improved YOLOv8-Based Algorithm for Classroom Behavior Recognition of Students：DMS-YOLOv8[J]. Computer Engineering and Applications, 2024, 60(24): 222-234.

参考文献

[1] 王文杰, 王有升. 评判与指引——弗兰德斯课堂互动分析系统与课堂互动评估系统的比较分析[J]. 教学研究, 2023, 46(5): 33-42.
WANG W J, WANG Y S. Evaluation and guidance: comparative analysis of Flanaders interaction analysis system and classroom assessment scoring system[J]. Teaching Research, 2023, 46(5): 33-42.
[2] 赖文天, 陈宇. 基于DPRNN语音分离的变时长型S-T课堂教学分析法[J]. 湖北第二师范学院学报, 2023, 40(2): 86-92.
LAI W T, CHEN Y. DPRNN based speech separation S-T classroom teaching analysis method with variable length[J]. Journal of Hubei Second Normal University, 2023, 40(2): 86-92.
[3] 董琪琪. 基于目标检测算法的学生课堂行为识别研究[D]. 天津: 河北工业大学, 2020.
DONG Q Q, Research on student classroom behavior recognition based on target detection algorithm[D]. Tianjin: Hebei University of Technology, 2020.
[4] 周叶. 基于Faster R-CNN的小学生课堂行为检测研究[D]. 成都: 四川师范大学, 2021.
ZHOU Y. Research on classroom behaviors detection of primary school students based on Faster R-CNN[D]. Chengdu: Sichuan Normal University, 2021.
[5] 王泽杰, 沈超敏, 赵春. 融合人体姿态估计和目标检测的学生课堂行为识别[J]. 华东师范大学学报 (自然科学版), 2022(2): 55-66.
WANG Z J, SHEN C M, ZHAO C. Recognition of classroom learning behaviors based on the fusion of human pose estimation and object detection[J]. Journal of East China Normal University (Natural Sciences), 2022(2): 55-66.
[6] 胡锦林, 齐永锋, 王佳颖. 基于时空图卷积网络的学生在线课堂行为识别[J]. 光电子·激光, 2022, 33(2): 149-156.
HU J L, QI Y F, WANG J Y. Recognition of students online classroom action based on spatiotemporal graph convolutional network[J]. Journal of Optoelectronics·Laser, 2022, 33(2): 149-156.
[7] 杨明远, 左栋. 基于改进YOLOV5算法的学生课堂行为识别研究[J]. 信息记录材料, 2022, 23(12): 51-53.
YANG M Y, ZUO D. Research on student classroom behavior recognition based on improved YOLOv5 algorithm[J]. Information Recording Materials, 2022, 23(12): 51-53.
[8] 张阳婷, 黄德启, 王东伟, 等. 基于深度学习的目标检测算法研究与应用综述[J]. 计算机工程与应用, 2023, 59(18): 1-13.
ZHANG Y T, HUANG D Q, WANG D W, et al Review on research and application of deep learning-based target detection algorithms[J]. Computer Engineering and Applications, 2023, 59(18): 1-13.
[9] BHARATI P, PRAMANIK A. Deep learning techniques—R-CNN to mask R-CNN: a survey[J]. Computational Intelligence in Pattern Recognition, 2020: 657-668.
[10] MAITY M, BANERJEE S, CHAUDHURI S S. Faster R-CNN and YOLO based vehicle detection: a survey[C]//Proceedings of the 2021 5th International Conference on Computing Methodologies and Communication (ICCMC), Erode, India, 2021: 1442-1447.
[11] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.
[12] OUZ E , KVVKMANISA A, DUVAR R, et al. A deep learning based fast lane detection approach[J]. Chaos, Solitons & Fractals, 2022, 155: 111722.
[13] YANG F, HUANG L, TAN X, et al. FasterNet-SSD: a small object detection method based on SSD model[J]. Signal, Image and Video Processing, 2024, 18(1): 173-180.
[14] LI X, LIU J, TANG Z, et al. MEDMCN: a novel multi-modal efficientdet with multi-scale capsnet for object detection[J]. The Journal of Supercomputing, 2024: 1-28.
[15] FAN P, ZHENG C, SUN J, et al. Enhanced real-time target detection for picking robots using lightweight centernet in complex orchard environments[J]. Agriculture, 2024, 14(7): 1059.
[16] HUSSAIN M. YOLOv1 to v8: unveiling each variant-a comprehensive review of YOLO[J]. IEEE Access, 2024, 12: 42816-42833.
[17] LAU K W, PO L M, REHMAN Y A U. Large separable kernel attention: rethinking the large kernel attention design in cnn[J]. Expert Systems with Applications, 2024, 236: 121352.
[18] GUO M H, LU C Z, LIU Z N, et al. Visual attention network[J]. Computational Visual Media, 2023, 9(4): 733-752.
[19] HOIEM D, DIVVALA S K, HAYS J H. Pascal VOC 2008 challenge[J]. World Literature Today, 2009, 24(1): 1-4.
[20] PADILLA R, NETTO S L, DA SILVA E A B. A survey on performance metrics for object-detection algorithms[C]//Proceedings of the 2020 International Conference on Systems, Signals and Image Processing (IWSSIP), Niteroi, Brazil, 2020: 237-242.
[21] ZHAO Y, KV W, XU S, et al. Detrs beat YOLOs on real-time object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, 2024: 16965-16974.
[22] TAN M, PANG R, LE Q V. EfficientDet: scalable and efficient object detection[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, 2020: 10781-10790.
[23] MUSHTAQ F, RAMESH K, DESHMUKH S, et al. Nuts & bolts: YOLO-v5 and image processing based component identification system[J]. Engineering Applications of Artificial Intelligence, 2023, 118: 105665.
[24] ATREY J, REGUNATHAN R, RAJASEKARAN R. Real-world application of face mask detection system using YOLOv6[J]. International Journal of Critical Infrastructures, 2024, 20(3): 216-240.
[25] 翟亚红, 王杰, 徐龙艳, 等. 基于改进YOLOv5n的舍养绵羊行为识别方法[J]. 农业机械学报, 2024, 55(4): 231-240.
ZHAI Y H, WANG J, XU L Y, et al. Behavior recognition of domesticated sheep based on improved YOLOv5n[J]. Transactions of the Chinese Society for Agricultural Machinery, 2024, 55(4): 231-240.
[26] 段青玲, 赵芷青, 蒋涛, 等. 基于SNSS-YOLO v7的肉牛行为识别方法[J]. 农业机械学报, 2023, 54(10): 266-274.
DUAN Q L, ZHAO Z Q, JIANG T, et al. Behavior recognition method of beef cattle based on SNSS-YOLOv7[J]. Transactions of the Chinese Society for Agricultural Machinery, 2023, 54(10): 266-274.
[27] 张立印, 张姬, 杨庆璐, 等. 基于视频和BCE-YOLO模型的奶牛采食行为检测[J]. 华南农业大学学报, 2024, 45(5): 782-792.
ZHANG L Y, ZHANG J, YANG Q L, et al. Detection of dairy cow feeding behavior based on video and BCE-YOLO model[J]. Journal of South China Agricultural University, 2024, 45(5): 782-792.