Computer Engineering and Applications ›› 2024, Vol. 60 ›› Issue (19): 32-45.DOI: 10.3778/j.issn.1002-8331.2403-0174
• Research Hotspots and Reviews • Previous Articles Next Articles
Cairang Zhuoma, LIU Pengfei, MA Guangxiang
Online:
2024-10-01
Published:
2024-09-30
才让卓玛,刘鹏飞,马光祥
Cairang Zhuoma, LIU Pengfei, MA Guangxiang. Survey of Infant Emotion Recognition[J]. Computer Engineering and Applications, 2024, 60(19): 32-45.
才让卓玛, 刘鹏飞, 马光祥. 婴儿情感识别综述[J]. 计算机工程与应用, 2024, 60(19): 32-45.
Add to citation manager EndNote|Ris|BibTeX
URL: http://cea.ceaj.org/EN/10.3778/j.issn.1002-8331.2403-0174
[1] 王卫平, 孙锟, 常立文, 等. 儿科学[M]. 9版. 北京:人民卫生出版社, 2018. WANG W P, SUN K, CHANG L W, et al. Pediatrics[M]. 9th ed. Beijing: People’s Medical Publishing House, 2018. [2] PETITTO L A, KATERELOS M, LEVY B G, et al. Bilingual signed and spoken language acquisition from birth: implications for the mechanisms underlying early bilingual language acquisition[J]. Journal of Child Language, 2001, 28(2): 453-496. [3] MALATESTA C Z, WILSON A. Emotion cognition interaction in personality development: a discrete emotions, functionalist analysis[J]. British Journal of Social Psychology, 1988, 27(1): 91-112. [4] DARWIN C. A biographical sketch of an infant[J]. Mind, 1877, 2(7): 285-294. [5] BRIDGES K M B. Emotional development in early infancy[J]. Child Development, 1932: 324-341. [6] HATFIELD L A, ELY E A. Measurement of acute pain in infants: a review of behavioral and physiological variables[J]. Biological Research for Nursing, 2015, 17(1): 100-111. [7] 徐瑄培, 张莉, 屈艺, 等. 婴儿啼哭声音的生理和病理因素分析[J]. 中国儿童保健杂志, 2018, 26(8): 869-872. XU X P, ZHANG L, QU Y, et al. Research progress on physiological and pathological factors of infant cry[J]. Chinese Journal of Child Health Care, 2018, 26(8): 869-872. [8] ZAMZMI G, KASTURI R, GOLDGOF D, et al. A review of automated pain assessment in infants: features, classification tasks, and databases[J]. IEEE Reviews in Biomedical Engineering, 2017, 11: 77-96. [9] JI C, MUDIYANSELAGE T B, GAO Y, et al. A review of infant cry analysis and classification[J]. EURASIP Journal on Audio, Speech, and Music Processing, 2021, 2021(1): 8. [10] JEYARAMAN S, MUTHUSAMY H, KHAIRUNIZAM W, et al. A review: survey on automatic infant cry analysis and classification[J]. Health and Technology, 2018, 8: 391-404. [11] FILIPPINI C, MERLA A. Systematic review of affective computing techniques for infant robot interaction[J]. International Journal of Social Robotics, 2023, 15(3): 393-409. [12] SCASSELLATI B, BRAWER J, TSUI K, et al. Teaching language to deaf infants with a robot and a virtual human[C]//Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 2018: 1-13. [13] KOKKONI E, MAVROUDI E, ZEHFROOSH A, et al. GEARing smart environments for pediatric motor rehabilitation[J]. Journal of Neuroengineering and Rehabilitation, 2020, 17: 1-15. [14] LI J, HASEGAWA-JOHNSON M, MCELWAIN N L. Analysis of acoustic and voice quality features for the classification of infant and mother vocalizations[J]. Speech Communication, 2021, 133: 41-61. [15] 程刚, 张大均, 关雨生, 等. 婴儿面孔表情标准化图片系统的初步建立[J]. 中国心理卫生杂志, 2015, 29(6): 406-412. CHENG G, ZHANG D J, GUAN Y S, et al. Preliminary establishment of the standardized Chinese infant facial expression of emotion[J]. Chinese Mental Health Journal, 2015, 29(6): 406-412. [16] NATHANI S, ERTMER D J, STARK R E. Assessing vocal development in infants and toddlers[J]. Clinical Linguistics & Phonetics, 2006, 20(5): 351-369. [17] BRAHNAM S, CHUANG C F, SEXTON R S, et al. Machine assessment of neonatal facial expressions of acute pain[J]. Decision Support Systems, 2007, 43(4): 1242-1254. [18] BRAHNAM S, NANNI L, MCMURTREY S, et al. Neonatal pain detection in videos using the iCOPEvid dataset and an ensemble of descriptors extracted from Gaussian of local descriptors[J]. Applied Computing and Informatics, 2023, 19(1/2): 122-143. [19] YAN J, LU G, LI X, et al. FENP: a database of neonatal facial expression for pain analysis[J]. IEEE Transactions on Affective Computing, 2020, 14(1): 245-254. [20] GRUNAU R E, OBERLANDER T, HOLSTI L, et al. Bedside application of the neonatal facial coding system in pain assessment of premature infants[J]. Pain, 1998, 76(3): 277-286. [21] WEBB R, AYERS S, ENDRESS A. The city infant faces database: a validated set of infant facial expressions[J]. Behavior Research Methods, 2018, 50: 151-159. [22] MAACK J K, BOHNE A, NORDAHL D, et al. The Tromso infant faces database (TIF): development, validation and application to assess parenting experience on clarity and intensity ratings[J]. Frontiers in Psychology, 2017, 8: 409. [23] REYES-GALAVIZ O F, CANO-ORTIZ S D, REYES-GARCÍA C A. Evolutionary-neural system to classify infant cry units for pathologies identification in recently born babies[C]//Proceedings of the 2008 Seventh Mexican International Conference on Artificial Intelligence, 2008: 330-335. [24] PLUTCHIK R. The psychology and biology of emotion[M].[S.l.]: HarperCollins College Publishers, 1994. [25] ACKERMAN B P, ABE J A A, IZARD C E. Differential emotions theory and emotional development: mindful of modularity[M]//What develops in emotional development. New York: Springer, 1998: 85-106. [26] PETITTO L A, HOLOWKA S, SERGIO L E, et al. Language rhythms in baby hand movements[J]. Nature, 2001, 413(6851): 35-36. [27] IZARD C E. The maximally discriminitive facial movements coding system, MAX[Z]. Newark :University of Delaware, 1979. [28] IZARD C E, DOUGHERTY L M, HEMBREE E A. A system for identifying affect expressions by holistic judgments (AFFEX)[Z]. Newark: University of Delaware. Instructional Resources Center, 1983. [29] DONDI M. Pre-Conference Workshop_Baby FACS: facial action coding system for infants and young children[C]//Proceedings of the International Congress of Infant Studies, 2022. [30] DUNSTAN P. Calm the crying: the secret baby language that reveals the hidden meaning behind an infant’s cry[M]. [S.l.]: Penguin, 2012. [31] ADDYMAN C, ADDYMAN I. The science of baby laughter[J]. Comedy Studies, 2013, 4(2): 143-153. [32] 程锐, 杨洋, 史源, 等. 新生儿疼痛评估与镇痛管理专家共识(2020版)[J]. 中国当代儿科杂志, 2020, 22(9): 923-930. CHENG R, YANG Y, SHI Y, et al. Expert consensus on neonatal pain assessment and analgesia management (2020 edition)[J]. Chinese Journal of Contemporary Pediatrics, 2020, 22(9): 923-930. [33] 乔建中. 情绪研究: 理论与方法[M]. 南京:南京师范大学出版社, 2003. QIAO J Z. Emotion research: theory and methods[M]. Nanjing: Nanjing Normal University Press, 2003. [34] EKMAN P, ROSENBERG E L.What the face reveals: basic and applied studies of spontaneous expression using the facial action coding system (FACS)[M]. New York: Oxford University Press, 1997. [35] HAMMAL Z, CHU W S, COHN J F, et al. Automatic action unit detection in infants using convolutional neural network[C]//Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII), 2017: 216-221. [36] EKMAN P, DAVIDSON R J, FRIESEN W V. The Duchenne smile: emotional expression and brain physiology: II[J]. Journal of Personality and Social Psychology, 1990, 58(2): 342. [37] ONAL ERTUGRUL I, AHN Y A, BILALPUR M, et al. Infant AFAR: automated facial action recognition in infants[J]. Behavior Research Methods, 2023, 55(3): 1024-1035. [38] OSTER H. Automated facial analysis of infant pain expressions: progress and future directions[J]. The Lancet Digital Health, 2021, 3(10): 613-614. [39] MAROULIS A. Baby FaceReader AU classification for infant facial expression configurations[J]. Measuring Behavior, 2018. [40] DEWI S P, PRASASTI A L, IRAWAN B. The study of baby crying analysis using MFCC and LFCC in different classification methods[C]//Proceedings of the 2019 IEEE International Conference on Signals and Systems (ICSigSys), 2019: 18-23. [41] BRAHNAM S, CHUANG C F, SHIH F Y, et al. Machine recognition and representation of neonatal facial displays of acute pain[J]. Artificial Intelligence in Medicine, 2006, 36(3): 211-222. [42] 卢官明, 李晓南, 李海波. 新生儿疼痛面部表情识别方法的研究[J]. 光学学报, 2008(11): 2109-2114. LU G M, LI X N, LI H B. Research on recognition for facial expression of pain in neonates[J]. Acta Optica Sinica, 2008(11): 2109-2114. [43] TIPPING M E. Sparse Bayesian learning and the relevance vector machine[J]. Journal of Machine Learning Research, 2001, 1: 211-244. [44] GHOLAMI B, HADDAD W M, TANNENBAUM A R. Relevance vector machine learning for neonate pain intensity assessment using digital imaging[J]. IEEE Transactions on Biomedical Engineering, 2010, 57(6): 1457-1466. [45] MANSOR M N, SYAM S H F, REJAB M N. AR model for infant pain anxiety recognition using fuzzy k-NN[C]//Proceedings of the 2012 International Symposium on Instrumentation & Measurement, Sensor Network and Automation (IMSNA), 2012: 374-376. [46] ZAMZAMI G, RUIZ G, GOLDGOF D, et al. Pain assessment in infants: towards spotting pain expression based on infants’ facial strain[C]//Proceedings of the 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), 2015: 1-5. [47] BAHDANAU D, CHO K, BENGIO Y. Neural machine translation by jointly learning to align and translate[J]. arXiv:1409.0473, 2014. [48] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv:2010.11929, 2020. [49] 乔欢欢. 婴儿表情识别算法研究[D]. 成都: 电子科技大学, 2019. QIAO H H. A research of infant expression recognition [D]. Chengdu: University of Electronic Science and Technology of China, 2019. [50] ZHANG L, XU C, LI S. Facial expression recognition of infants based on multi-stream CNN fusion network[C]//Proceedings of the 2020 IEEE 5th International Conference on Signal and Image Processing (ICSIP), 2020: 37-41. [51] 邓源, 施一萍, 江悦莹, 等. 基于MobileNetV2与LBP特征融合的婴幼儿表情识别算法[J]. 电子科技, 2022, 35(8): 47-52. DENG Y, SHI Y P, JIANG Y Y, et al. Infant expression recognition algorithm based on MobileNetV2 and LBP feature fusion[J]. Electronic Science and Technology, 2022, 35(8): 47-52. [52] LIN Q, HE R, JIANG P. Feature guided CNN for baby’s facial expression recognition[J]. Complexity, 2020, 2020: 1-10. [53] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2018: 7132-7141. [54] YAN S, ZHENG W, TANG C, et al. ARL-IL CNN for automatic facial expression recognition of infants under 24 months of age[J]. Journal of Physics: Conference Series, 2020, 1518(1): 012027. [55] CAI J, MENG Z, KHAN A S, et al. Island loss for learning discriminative features in facial expression recognition[C]//Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), 2018: 302-309. [56] 耿磊, 齐婷婷, 张芳, 等. 基于MIFNet的婴儿面部表情识别 [J]. 计算机系统应用, 2023, 32(8): 42-53. GENG L, QI T T, ZHANG F, et al. Facial expression recognition of infants based on MIFNet [J]. Computer Systems & Applications, 2023, 32(8): 42-53. [57] LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021: 10012-10022. [58] LI X, WANG W, HU X, et al. Selective kernel networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019: 510-519. [59] LU G, CHEN H, WEI J, et al. Video-based neonatal pain expression recognition with cross-stream attention[J]. Multimedia Tools and Applications, 2024, 83(2): 4667-4690. [60] LI C, LI X, LI T. Infant facial expression recognition based on parameter-free attention module[C]//Proceedings of the 2022 International Conference on Artificial Intelligence and Computer Information Technology (AICIT), 2022: 1-4. [61] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the European Conference on Computer Vision (ECCV), 2018: 3-19. [62] LI C, POURTAHERIAN A, VAN ONZENOORT L, et al. Infant facial expression analysis: towards a real-time video monitoring system using R-CNN and HMM[J]. IEEE Journal of Biomedical and Health Informatics, 2020, 25(5): 1429-1440. [63] NING M, ERTUGRUL I O, MESSINGER D S, et al. Automated emotional valence estimation in infants with stochastic and strided temporal sampling[C]//Proceedings of the 2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII), 2023: 1-8. [64] TONG Z, SONG Y, WANG J, et al. VideoMAE: masked autoencoders are data-efficient learners for self-supervised video pre-training[C]//Advances in Neural Information Processing Systems, 2022, 35: 10078-10093. [65] BANO S, RAVIKUMAR K M. Decoding baby talk: a novel approach for normal infant cry signal classification[C]//Proceedings of the 2015 International Conference on Soft-Computing and Networks Security (ICSNS), 2015: 1-4. [66] FELIPE G Z, AGUIAR R L, COSTA Y M G, et al. Identification of infants’ cry motivation using spectrograms[C]//Proceedings of the 2019 International Conference on Systems, Signals and Image Processing (IWSSIP), 2019: 181-186. [67] CHANG C Y, BHATTACHARYA S, RAJ VINCENT P M D, et al. An efficient classification of neonates cry using extreme gradient boosting-assisted grouped-support-vector network[J]. Journal of Healthcare Engineering, 2021, 2021(1): 7517313. [68] DEHAK N, KENNY P J, DEHAK R, et al. Front-end factor analysis for speaker verification[J]. IEEE Transactions on Audio, Speech, and Language Processing, 2010, 19(4): 788-798. [69] BĂNICĂ I A, CUCU H, BUZO A, et al. Automatic methods for infant cry classification[C]//Proceedings of the 2016 International Conference on Communications (COMM), 2016: 51-54. [70] SHARMA K, GUPTA C, GUPTA S. Infant weeping calls decoder using statistical feature extraction and Gaussian mixture models[C]//Proceedings of the 2019 10th International Conference on Computing, Communication and Networking Technologies (ICCCNT), 2019: 1-6. [71] PETRONI M, MALOWANY A S, JOHNSTON C C, et al. Classification of infant cry vocalizations using artificial neural networks (ANNs)[C]//Proceedings of the 1995 International Conference on Acoustics, Speech, and Signal Processing, 1995: 3475-3478. [72] FRANTI E, ISPAS I, DASCALU M. Testing the universal baby language hypothesis-automatic infant speech recognition with CNNs[C]//Proceedings of the 2018 41st International Conference on Telecommunications and Signal Processing (TSP), 2018: 1-4. [73] LE L, KABIR A N M H, JI C, et al. Using transfer learning, SVM, and ensemble classification to classify baby cries based on their spectrogram images[C]//Proceedings of the 2019 IEEE 16th International Conference on Mobile Ad Hoc and Sensor Systems Workshops (MASSW), 2019: 106-110. [74] 谢湘, 张立强, 王晶. 残差网络在婴幼儿哭声识别中的应用 [J]. 电子与信息学报, 2019, 41(1): 233-239. XIE X, ZHANG L Q, WANG J. Application of residual network to infant crying recognition[J]. Journal of Electronics & Information Technology, 2019, 41(1): 233-239. [75] OZSEVEN T. Infant cry classification by using different deep neural network models and hand-crafted features[J]. Biomedical Signal Processing and Control, 2023, 83: 104648. [76] MAGHFIRA T N, BASARUDDIN T, KRISNADHI A. Infant cry classification using CNN-RNN[J]. Journal of Physics: Conference Series, 2020, 1528(1): 012019. [77] 余莉萍, 梁镇麟, 梁瑞宇. 基于改进LSTM的儿童语音情感识别模型 [J]. 计算机工程, 2020, 46(6): 40-49. YU L P, LIANG Z L, LIANG R Y. Emotion recognition model for children speech based on improved LSTM[J]. Computer Engineering, 2020, 46(6): 40-49. [78] 李彬. 基于图卷积神经网络和Transformer表示方法的婴儿哭声分类方法研究 [D]. 合肥:安徽大学, 2022. LI B. Research on infant cry classification method based on graph convolutional neural network and Transformer representation[D]. Hefei: Anhui University, 2022. [79] QIAO X, JIAO S, LI H, et al. Infant cry classification using an efficient graph structure and attention-based model[J]. Kuwait Journal of Science, 2024, 51(3): 100221. [80] XU H, ZHANG J, DAI L. Differential time-frequency log-mel spectrogram features for vision transformer based infant cry recognition[C]//Proceedings of the Interspeech 2022, 2022: 1963-1967. [81] MAJUMDER N, HAZARIKA D, GELBUKH A, et al. Multimodal sentiment analysis using hierarchical fusion with context modeling[J]. Knowledge-Based Systems, 2018, 161: 124-133. [82] 程大雷, 张代玮, 陈雅茜. 多模态情感识别综述[J]. 西南民族大学学报 (自然科学版), 2022, 48(4): 440-447. CHENG D L, ZHANG D W, CHEN Y X. Survey on multimodal emotion recognition[J]. Journal of Southwest Minzu University(Natural Science Edition), 2022, 48(4): 440-447. [83] PAL P, IYER A N, YANTORNO R E. Emotion detection from infant facial expressions and cries[C]//Proceedings of the 2006 IEEE International Conference on Acoustics Speech and Signal Processing, 2006. [84] XU L, KRZYZAK A, SUEN C Y. Methods of combining multiple classifiers and their applications to handwriting recognition[J]. IEEE Transactions on Systems, Man, and Cybernetics, 1992, 22(3): 418-435. [85] FANG C Y, MA C W, CHIANG M L, et al. An infant emotion recognition system using visual and audio information[C]//Proceedings of the 2017 4th International Conference on Industrial Engineering and Applications (ICIEA), 2017: 284-291. |
No related articles found! |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||