Review of Research on Driver Fatigue Driving Detection Methods

doi:10.3778/j.issn.1002-8331.2204-0053

Abstract

Abstract: The proportion of traffic accidents caused by fatigue driving has increased year by year, which has attracted widespread attention from researchers. At present, the research of fatigue driving testing is limited by various factors such as scientific and technological level, environment, and road, which makes it difficult to further develop fatigue driving detection technology. This article introduces the latest progress in driver fatigue driving detection methods in the past decade. The two categories of active detection method and passive detection method are elaborated and reviewed. According to the different characteristics of the two major types of detection methods, it is carefully classified. The advantages and limitations of various fatigue driving detection methods are further analyzed, and the detection algorithms used in the active detection method based on facial features in the past three years are analyzed and summarized. Finally, the shortcomings of various fatigue driving detection methods are summarized, and the future research trends in the field of fatigue detection are proposed, which provides new ideas for researchers to further research.

Key words: fatigue driving, subjective detection method, objective detection method, detecting algorithm

摘要： 由于疲劳驾驶导致的交通事故占比逐年上涨，引起了研究人员的广泛关注。目前疲劳驾驶检测的研究受限于科技水平、环境、道路等各种因素的影响，导致疲劳驾驶检测技术难以进一步发展。介绍了近10年内驾驶员疲劳驾驶检测方法的最新进展。阐述并回顾了主动检测法和被动检测法两大类。根据两大类检测方法各自不同的特征进行细致的分类。进一步分析了各类疲劳驾驶检测方法的优势和局限，同时对主动检测法中基于面部特征的检测方法近3年内所使用的检测算法进行了分析和总结。归纳了各类疲劳驾驶检测方法存在的不足，同时提出疲劳检测领域未来的研究趋势，为研究人员进一步的研究提供新的思路。

关键词: 疲劳驾驶, 主动检测法, 被动检测法, 检测算法

ZHANG Rui, ZHU Tianjun, ZOU Zhiliang, SONG Rui. Review of Research on Driver Fatigue Driving Detection Methods[J]. Computer Engineering and Applications, 2022, 58(21): 53-66.

张瑞, 朱天军, 邹志亮, 宋瑞. 驾驶员疲劳驾驶检测方法研究综述[J]. 计算机工程与应用, 2022, 58(21): 53-66.

References

[1] 李都厚，刘群，袁伟，等.疲劳驾驶与交通事故关系[J].交通运输工程学报，2010，10（2）：104-109.
LI D H，LIU Q，YUAN W，et al.Relationship between fatigue driving and traffic accident[J].Journal of Traffic and Transportation Engineering，2010，10（2）：104-109.
[2] KHUMPISUTH O，CHOTCHINASRI T，KOSHAKOSAI V，et al.Driver drowsiness detection using eye-closeness detection[C]//2016 12th International Conference on Signal-Image Technology & Internet-Based Systems（SITIS），2016：661-668.
[3] KOH S，CHO B R，LEE J，et al.Driver drowsiness detection via PPG biosignals by using multimodal head support[C]//2017 4th International Conference on Control，Decision and Information Technologies（CoDIT），2017：383-388.
[4] HOTTA Y，ITO K.EMG-based detection of muscle fatigue during low-level isometric contraction：effects of electrode configuration and blood flow restriction[C]//2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society，2011：3877-3879.
[5] ZHANG Y Q，ZHENG W L，LU B L.Transfer components between subjects for EEG-based driving fatigue detection[C]//International Conference on Neural Information Processing.Cham：Springer，2015：61-68.
[6] LI M，ZHANG C，YANG J F.An EEG-based method for detecting drowsy driving state[C]//2010 Seventh International Conference on Fuzzy Systems and Knowledge Discovery，2010：2164-2167.
[7] 周展鹏，孔万增，王奕直，等.基于心电和脑电的驾驶疲劳检测研究[J].杭州电子科技大学学报，2014，34（3）：25-28.
ZHOU Z P，KONG W Z，WANG Y Z，et al.ECG and EEG based detection of driver fatigue[J].Journal of Hangzhou Dianzi University，2014，34（3）：25-28.
[8] LAL S K L，CRAIG A.A critical review of the psychophysiology of driver fatigue[J].Biological Psychology，2001，55（3）：173-194.
[9] JIANG K，LING F，FENG Z，et al.Why do drivers continue driving while fatigued？An application of the theory of planned behaviour[J].Transportation Research Part A：Policy and Practice，2017，98：141-149.
[10] BENER A，YILDIRIM E，?ZKAN T，et al.Driver sleepiness，fatigue，careless behavior and risk of motor vehicle crash and injury：population based case and control study[J].Journal of Traffic and Transportation Engineering（English Edition），2017，4（5）：496-502.
[11] NOUGHABI M G，SADEGHI A，MOGHADDAM A M，et al.Fatigue risk management：assessing and ranking the factors affecting the degree of fatigue and sleepiness of heavy-vehicle drivers using TOPSIS and statistical analyses[J].Iranian Journal of Science and Technology，Transactions of Civil Engineering，2020，44（4）：1345-1357.
[12] 杜振君.基于工效学的某汽车总装车间叉车司机疲劳评估研究[D].沈阳：沈阳工业大学，2020.
DU Z J.Study on fatigue assessment of forklift driver in ageneral a ssembly workshop based on ergonomics[D].Shenyang：Shenyang University of Technology，2020.
[13] LEE S，KIM J K.Factors contributing to the risk of airline pilot fatigue[J].Journal of Air Transport Management，2018，67：197-207.
[14] FILTNESS A J，NAWEED A.Causes，consequences and counter measures to driver fatigue in the rail industry：the train driver perspective[J].Applied Ergonomics，2017，60：12-21.
[15] 李响，徐玉萍，章海亮.机车司机驾驶疲劳风险动态量化评价研究[J].中国安全科学学报，2017，27（2）：18-23.
LI X，XU Y P，ZHANG H L.A study on dynamic quantitative evaluation of train driver’s fatigue risk[J].China Safety Science Journal，2017，27（2）：18-23.
[16] MOLLICONE D，KAN K，MOTT C，et al.Predicting performance and safety based on driver fatigue[J].Accident Analysis & Prevention，2019，126：142-145.
[17] LI M K，YU J J，MA L，et al.Modeling and mitigating fatigue-related accident risk of taxi drivers[J].Accident Analysis & Prevention，2019，123：79-87.
[18] POMERLEAU D.RALPH：rapidly adapting lateral position handler[C]//Proceedings of the Intelligent Vehicles’95，1995：506-511.
[19] 郭思强，滕靖，郭旭健，等.基于车辆行驶数据的营运车驾驶员疲劳驾驶监控研究[C]//2014第九届中国智能交通年会，2014：91-102.
GUO S Q，TENG J，GUO X J，et al.Research on fatigue driving of commercial vehicle drivers based on multi-source real data[C]//Proceedings of the 9th China Intelligent Transportation Annual Conference，2014：91-102.
[20] 黄皓.基于驾驶操作及车辆状态的疲劳驾驶行为检测研究[D].南京：东南大学，2016.
HUANG H.Fatigue driving detection based on driver behavior and vehicle state[D].Nanjing：Southeast University，2016.
[21] 蔡素贤，杜超坎，周思毅，等.基于车辆运行数据的疲劳驾驶状态检测[J].交通运输系统工程与信息，2020，20（4）：77-82.
CAI S X，DU C K，ZHOU S Y，et al.Fatigue driving state detection based on vehicle running data[J].Journal of Transportation Systems Engineering and Information Technology，2020，20（4）：77-82.
[22] RIERA L，OZCAN K，MERICKEL J，et al.Detecting and tracking unsafe lane departure events for predicting driver safety in challenging naturalistic driving data[C]//2020 IEEE Intelligent Vehicles Symposium（IV），2020：238-245.
[23] CHEN L W，CHEN H M.Driver behavior monitoring and warning with dangerous driving detection based on the internet of vehicles[J].IEEE Transactions on Intelligent Transportation Systems，2020，22（11）：7232-7241.
[24] 张希波，成波，冯睿嘉.基于方向盘操作的驾驶人疲劳状态实时检测方法[J].清华大学学报（自然科学版），2010，50（7）：1072-1076.
ZHANG X B，CHENG B，FENG R J.Real-time detection of driver drowsiness based on steering performance[J].Journal of Tsinghua University（Science and Technology），2010，50（7）：1072-1076.
[25] 沙春发，李瑞，张明明.基于方向盘握力的疲劳驾驶检测研究[J].科学技术与工程，2016，16（30）：299-304.
SHA C F，LI R，ZHANG M M.Detecting fatigue driving based on steering wheel grip force[J].Science Technology and Engineering，2016，16（30）：299-304.
[26] MCDONALD A D，LEE J D，SCHWARZ C，et al.A contextual and temporal algorithm for driver drowsiness detection[J].Accident Analysis & Prevention，2018，113：25-37.
[27] LI Z，CHEN L，NIE L，et al.A novel learning model of driver fatigue features representation for steering wheel angle[J].IEEE Transactions on Vehicular Technology，2021，71（1）：269-281.
[28] LI R，CHEN Y V，ZHANG L.A method for fatigue detection based on driver’s steering wheel grip[J].International Journal of Industrial Ergonomics，2021，82：103083.
[29] 彭军强，吴平东，殷罡.疲劳驾驶的脑电特性探索[J].北京理工大学学报，2007，27（7）：585-589.
PENG J Q，WU P D，YIN G.Explorating the characters of electroencephalogram for fatigued drivers[J].Transactions of Beijing Institute of Technology，2007，27（7）：585-589.
[30] LAL S K L，CRAIG A.Driver fatigue：electroencephalography and psychological assessment[J].Psychophysiology，2002，39（3）：313-321.
[31] HOUSHMAND S，KAZEMI R，SALMANZADEH H.A novel convolutional neural network method for subject-independent driver drowsiness detection based on single-channel data and EEG alpha spindles[J].Proceedings of the Institution of Mechanical Engineers，Part H：Journal of Engineering in Medicine，2021，235（9）：1069-1078.
[32] JEONG J H，YU B W，LEE D H，et al.Classification of drowsiness levels based on a deep spatio-temporal convolutional bidirectional LSTM network using electroencephalography signals[J].Brain Sciences，2019，9（12）：348.
[33] ZOU S，QIU T，HUANG P，et al.Constructing multi-scale entropy based on the empirical mode decomposition（EMD） and its application in recognizing driving fatigue[J].Journal of Neuroscience Methods，2020，341：108691.
[34] CHINARA S.Automatic classification methods for detecting drowsiness using wavelet packet transform extracted time-domain features from single-channel EEG signal[J].Journal of Neuroscience Methods，2021，347：108927.
[35] LIU Q，LIU Y，CHEN K，et al.Research on channel selection and multi-feature fusion of EEG signals for mental fatigue detection[J].Entropy，2021，23（4）：457.
[36] ZHENG W L，LU B L.A multimodal approach to estimating vigilance using EEG and forehead EOG[J].Journal of Neural Engineering，2017，14（2）：026017.
[37] GROMER M，SALB D，WALZER T，et al.ECG sensor for detection of driver’s drowsiness[J].Procedia Computer Science，2019，159：1938-1946.
[38] 徐礼胜，张闻勖，庞宇轩，等.基于短时心电信号的疲劳驾驶检测算法[J].东北大学学报（自然科学版），2019，40（7）：937-941.
XU L S，ZHANG W X，PANG Y X，et al.Driver drowsiness detection algorithm using short-term ECG signals[J].Journal of Northeastern University（Natural Science），2019，40（7）：937-941.
[39] MURUGAN S，SELVARAJ J，SAHAYADHAS A.Detection and analysis：driver state with electrocardiogram（ECG）[J].Physical and Engineering Sciences in Medicine，2020，43（2）：525-537.
[40] LEE H，LEE J，SHIN M.Using wearable ECG/PPG sensors for driver drowsiness detection based on distinguishable pattern of recurrence plots[J].Electronics，2019，8（2）：192.
[41] SALVATI L，D’AMORE M，FIORENTINO A，et al.On-road detection of driver fatigue and drowsiness during medium-distance journeys[J].Entropy，2021，23（2）：135.
[42] WANG H.Detection and alleviation of driving fatigue based on EMG and EMS/EEG using wearable sensor[C]//Proceedings of the 5th EAI International Conference on Wireless Mobile Communication and Healthcare，2015：155-157.
[43] FAN Y，GU F，WANG J，et al.SafeDriving：an effective abnormal driving behavior detection system based on EMG signals[J].IEEE Internet of Things Journal，2021，9（14）：12338-12350.
[44] WIERWILLE W，KNIPLING R.Vehicle-based drowsy driver detection：current status and future prospects[C]//Proceedings of the IVHS America Conference，1994：245-256.
[45] DINGES D F，GRACE R.PERCLOS：a valid psychophysiological measure of alertness as assessed by psychomotor vigilance[D].US Department of Transportation.Federal Highway Administration，Publication Number FHWA-MCRT-98-006，1998.
[46] TAO H，ZHANG G，ZHAO Y，et al.Real-time driver fatigue detection based on face alignment[C]//Ninth International Conference on Digital Image Processing（ICDIP 2017），2017：6-11.
[47] 潘志庚，刘荣飞，张明敏.基于模糊综合评价的疲劳驾驶检测算法研究[J].软件学报，2019，30（10）：2954-2963.
PAN Z G，LIU R F，ZHANG M M.Research on fatigue driving detection algorithm based on fuzzy comprehensive evaluation[J].Journal of Software，2019，30（10）：2954-2963.
[48] ZHUANG Q，KEHUA Z，WANG J，et al.Driver fatigue detection method based on eye states with pupil and iris segmentation[J].IEEE Access，2020，8：173440-173449.
[49] BAKHEET S，AL-HAMADI A.A framework for instantaneous driver drowsiness detection based on improved HOG features and Na?ve Bayesian classification[J].Brain Sciences，2021，11（2）：240.
[50] WANG R B，GOU L，TONG B L，et al.Monitoring mouth movement for driver fatigue or distraction with one camera[C]//The 7th International IEEE Conference on Intelligent Transportation Systems（IEEE Cat.No.04TH8749），2004：314-319.
[51] ANITHA C，VENKATESHA M K，ADIGA B S.A two fold expert system for yawning detection[J].Procedia Computer Science，2016，92：63-71.
[52] JIE Z，MAHMOUD M，STAFFORD-FRASER Q，et al.Analysis of yawning behaviour in spontaneous expressions of drowsy drivers[C]//2018 13th IEEE International Conference on Automatic Face & Gesture Recognition（FG 2018），2018：571-576.
[53] ADHINATA F D，RAKHMADANI D P，WIJAYANTO D.Fatigue detection on face image using FaceNet algorithm and K-nearest neighbor classifier[J].Journal of Information Systems Engineering and Business Intelligence，2021，7（1）：22-30.
[54] 李勇达，张超，孟令君.基于头部姿态特征的列车司机疲劳驾驶检测系统研究[J].交通信息与安全，2014，32（5）：114-119.
LI Y D，ZHANG C，MENG L J.A fatigue driving detection system for train driver based on head pose features[J].Journal of Transport Information and Safety，2014，32（5）：114-119.
[55] RUIZ N，CHONG E，REHG J M.Fine-grained head pose estimation without keypoints[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops，2018：2074-2083.
[56] ANSARI S，NAGHDY F，DU H，et al.Driver mental fatigue detection based on head posture using new modified reLU-BiLSTM deep neural network[J].IEEE Transactions on Intelligent Transportation Systems，2021.
[57] ZHANG N，ZHANG H，HUANG J.Driver fatigue state detection based on facial key points[C]//2019 6th International Conference on Systems and Informatics（ICSAI），2019：144-149.
[58] 刘鹏，江朝晖，熊进，等.用于驾驶疲劳检测的人眼定位及状态判别算法[J].计算机工程与应用，2010，46（24）：185-188.
LIU P，JIANG Z H，XIONG J，et al.Algorithm of eye localization and eye state recognition in driving fatigue detection[J].Computer Engineering and Applications，2010，46（24）：185-188.
[59] YOU F，LI X，GONG Y，et al.A real-time driving drowsiness detection algorithm with individual differences consideration[J].IEEE Access，2019，7：179396-179408.
[60] ZHANG F，WANG F.Exercise fatigue detection algorithm based on video image information extraction[J].IEEE Access，2020，8：199696-199709.
[61] LIU W，QIAN J，YAO Z，et al.Convolutional two-stream network using multi-facial feature fusion for driver fatigue detection[J].Future Internet，2019，11（5）：115.
[62] JI Y，WANG S，ZHAO Y，et al.Fatigue state detection based on multi-index fusion and state recognition network[J].IEEE Access，2019，7：64136-64147
[63] ZHAO Z，ZHOU N，ZHANG L，et al.Driver fatigue detection based on convolutional neural networks using EM-CNN[J].Computational Intelligence and Neuroscience，2020.
[64] LI K，GONG Y，REN Z.A fatigue driving detection algorithm based on facial multi-feature fusion[J].IEEE Access，2020，8：101244-101259.
[65] YOU F，GONG Y，TU H，et al.A fatigue driving detection algorithm based on facial motion information entropy[J].Journal of Advanced Transportation，2020.
[66] LIU Z，PENG Y，HU W.Driver fatigue detection based on deeply-learned facial expression representation[J].Journal of Visual Communication and Image Representation，2020，71：102723.
[67] 张伟，黄炜，罗大庸.基于多特征量贝叶斯融合的驾驶疲劳识别[J].计算机工程与应用，2012，48（33）：244-248.
ZHANG W，HUANG W，LUO D Y.Recognition of driver fatigue using multi-feature fusion by Bayesian network[J].Computer Engineering and Applications，2012，48（33）：244-248.
[68] ZHU T，ZHANG C，WU T，et al.Research on a real-time driver fatigue detection algorithm based on facial video sequences[J].Applied Sciences，2022，12（4）：2224.
[69] HONG S，KWON H，CHOI S H，et al.Intelligent system for drowsiness recognition based on ear canal electroencephalography with photoplethysmography and electrocardiography[J].Information Sciences，2018，453：302-322.
[70] VARDHAN V V S，REDDY N R K，SURYA K J，et al.Driver’s drowsiness detection based on facial multi-feature fusion[C]//Journal of Physics：Conference Series，2021，1998（1）：012034.
[71] JIA H，XIAO Z，JI P.Fatigue driving detection based on deep learning and multi-index fusion[J].IEEE Access，2021，9：147054-147062.
[72] DU G，LI T，LI C，et al.Vision-based fatigue driving recognition method integrating heart rate and facial features[J].IEEE Transactions on Intelligent Transportation Systems，2020，22（5）：3089-3100.
[73] LI K，WANG S，DU C，et al.Accurate fatigue detection based on multiple facial morphological features[J].Journal of Sensors，2019.
[74] ZHAO Y，XIE K，ZOU Z，et al.Intelligent recognition of fatigue and sleepiness based on InceptionV3-LSTM via multi-feature fusion[J].IEEE Access，2020，8：144205-144217.