Sound Event Localization and Detection Based on Dual Attention

doi:10.3778/j.issn.1002-8331.2206-0200

Abstract

Abstract: In recent years, sound event localization and detection have been widely used in various fields. The network model of sound event localization and detection based on deep learning is difficult to accurately capture the spatial and channel information of the input feature map, which leads to the difficulty of sound event localization and detection. An attention-based CECANet（coordinate and efficient channel attention network） network model is proposed. Firstly, a coordinate attention module is introduced into the residual module to make the network model pay more attention to the spatial coordinate information of the feature map, and then an efficient channel attention module is added after the average pooling layer to make the network model pay more attention to the channel information between features. The experimental results show that the proposed network model in the TAU-NIGENS Spatial Sound Events 2021 dataset has an overall improvement in performance compared to the baseline model, with F1 and LR improved to 0.720 and 0.728, and ER and LE reduced to 0.393 and 11.71°.

Key words: sound event localization and detection, attention mechanism, convolutional neural network, deep learning

摘要： 近年来，声音事件定位与检测被广泛应用于各个领域。基于深度学习的声音事件定位与检测的网络模型难以准确捕捉输入特征图的空间和通道信息，从而导致声音事件定位和检测难度较大。提出了一种基于注意力的CECANet（coordinate and efficient channel attention network）网络模型。在残差模块中引入坐标注意力模块，使网络模型更集中关注特征图的空间坐标信息，然后在平均池化层后加入高效通道注意力模块，使网络模型更加关注特征之间的通道信息。实验结果表明，提出的网络模型在TAU-NIGENS Spatial Sound Events 2021数据集中，相较于基线模型性能有整体的提升，F1和LR提升到了0.720和0.728，ER和LE降低到0.393和11.71°。

关键词: 声音事件定位与检测, 注意力机制, 卷积神经网络, 深度学习

XU Chundong, LIU Hao, MIN Yuan, ZHEN Yadi. Sound Event Localization and Detection Based on Dual Attention[J]. Computer Engineering and Applications, 2023, 59(19): 99-105.

许春冬, 刘昊, 闵源, 甄雅迪. 基于双重注意力的声音事件定位与检测[J]. 计算机工程与应用, 2023, 59(19): 99-105.

References

[1] DABRAN I，ELMAKIAS O，SHMELKIN R，et al.An intelligent sound ALAM recognition system for smart cars and smart homes[C]//Proceeding of the 2018 IEEE/IFIP Network Operations and Management Symposium，2018：1-4.
[2] LI Y，LI X，ZHANG Y，et al.Anomalous sound detection using deep audio representation and a BLSTM network for audio surveillance of roads[J].IEEE Access，2018，6：58043-58055.
[3] CAKIR E，ADAVANNE S，PARASCANDOLO G，et al.Convolutional recurrent neural networks for bird audio detection[C]//Proceeding of the 25th European Signal Processing Conference，2017：1744-1748.
[4] MARQUES T A，THOMAS L，MARTIN S W，et al.Estimating animal population density using passive acoustics[J].Biological Reviews，2013，88（2）：287-309.
[5] HUMAYUN A I，GHAFFARZADEGAN S，FENG Z，et al.Learning front-end filter-bank parameters using convolutional neural networks for abnormal heart sound detection[C]//Proceeding of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society，2018：1408-1411.
[6] SCHR?DER J，MORITZ N，SCH?DLER M R，et al.On the use of Spectro-temporal features for the IEEE AASP challenge “detection and classification of acoustic scenes and events”[C]//Proceeding of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics，2013：1-4.
[7] HEITTOLA T，MESAROS A，ERONEN A，et al.Context-dependent sound event detection[J].EURASIP Journal on Audio，Speech，and Music Processing，2013（1）：1-13.
[8] KOMATSU T，TOIZUMI T，KONDO R，et al.Acoustic event detection method using semi-supervised nonnegative matrix factorization with mixtures of local dictionaries[C]//Proceedings of the Detection and Classification of Acoustic Scenes and Events 2016 Workshop（DCASE 2016），2016：45-49.
[9] XU X，YU J，CHEN Y，et al.Leveraging audio signals for early recognition of inattentive driving with smartphones[J].IEEE Transactions on Mobile Computing，2017，17（7）：1553-1567.
[10] VELáZQUEZ I M，REN Y，HANEDA Y，et al.A fusion method based on class rotations for DNN-DoA estimation on spherical microphone array[C]//Proceeding of the 29th European Signal Processing Conference（EUSIPCO），2021：885-889.
[11] 鄢社锋，马远良，侯朝焕.宽带波束域相干信号子空间高分辨方位估计[J].声学学报，2006（5）：418-424.
YAN S F，MA Y L，HOU Z H.Broadband highresolution direction-of-arrival estimation using beamspace coherent signal subspace approach[J].Acta Acustica，2006（5）：418-424.
[12] 李伟红，汤海兵，龚卫国.公共场所异常声源定位中时延估计方法研究[J].仪器仪表学报，2012，33（4）：750-756.
LI W H，TANG H B，GONG W G.Time delay estimation method of abnormal sound source localization in public places[J].Chinese Journal of Scientific Instrument，2012，33（4）：750-756..
[13] BATTAGLINO D，LEPAULOUX L，EVANS N，et al.Acoustic scene classification using convoluteonal neural networks[J].IEEE AASP Challenge on Detec，2016.
[14] ZINEMANAS P，CANCELA P，ROCAMORA M.End-to-end convolutional neural networks for soundevent detection in urban environments[C]//Proceeding of the 24th Conference of Open Innovations Association（FRUCT），2019：533-539.
[15] HAYASHI T，WATANABE S，TODA T，et al.Duration-controlled LSTM for polyphonic sound event detection[J].IEEE/ACM Transactions on Audio，Speech，and Language Processing，2017，25（11）：2059-2070.
[16] Z?HRER M，PERNKOPF F.Virtual adversarial training，and data augmentation for acoustic event detection with gated recurrent neural networks[C]//Proceeding of Interspeech 2017，2017：493-497.
[17] PHAN H，CHéN O Y，KOCH P，et al.Unifyingisolated and overlapping audio event detection with multi-label multi-task convolutional recurrent neural networks[C]//Proceeding of 2019 IEEE International Conference on Acoustics，Speech and Signal Processing，2019：51-55.
[18] JUNG S，PARK J，LEE S.Polyphonic sound event detection using convolutional bidirectional LSTM and synthetic data-based transfer learning[C]//Proceeding of 2019 IEEE International Conference on Acoustics，Speech and Signal Processing，2019：885-889.
[19] ZHANG Y，WANG S，LI Z，et al.Data augmentation and class-based ensembled CNN-conformer networks for sound event localization and detection[R].Technical Report of DCASE Challenge，2021.
[20] LEE S H，HWANG J W，SEO S B，et al.Sound event localization and detection using cross-modal attention and parameter sharing for DCASE2021 challenge[R].Technical Report of DCASE Challenge，2021.
[21] 高广尚.深度学习推荐模型中的注意力机制研究综述[J].计算机工程与应用，2022，58（9）：9-18.
GAO G S.Survey on attention mechanism in deep learning recommendation models[J].Computer Engineering and Applications，2022，58（9）：9-18.
[22] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017.
[23] NGUYEN T N T，JONES D L，WATCHARASUPAT K N，et al.SALSA-Lite：a fast and effective feature for polyphonic sound event localization and detection with microphone arrays[C]//Proceeding of 2022 IEEE International Conference on Acoustics，Speech and Signal Processing，2022：716-720.
[24] HU J，LI S，SUN G，et al.Squeeze-and-excitation networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[25] HOU Q，ZHOU D，FENG J.Coordinate attention for efficient mobile network design[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition，2021：13713-13722.
[26] WANG Q L，WU B G，ZHU P F，et al.ECA-Net：efficient channel attention for deep convolutional neural networks[C]//Proceedings of the CVF Conference on Computer Vision and Pattern Recognition，2020：11531-11539.
[27] POLITIS A，ADAVANNE S，KRAUSE D，et al.A dataset of dynamic reverberant sound scenes with directional interferers for sound event l-ocalization and detection[J].arXiv：2106.06999，2021.
[28] SHIMADA K，KOYAMA Y，TAKAHASHI N，et al.Accdoa：activity?coupled cartesian direction of arrival representation for sound event loc-alization and detection[C]//Proceeding of 2021 IEEE International Conference on Acoustics，Speech and Signal Processing，2021：915-919.