Multi-Scale End-to-End Speaker Recognition System Based on Improved Res2Net

doi:10.3778/j.issn.1002-8331.2208-0085

Abstract

Abstract: The feature extraction ability of lightweight convolutional neural networks in speaker recognition systems is weak and recognition is poor. And to improve the feature extraction ability, many methods use deeper, wider and more complex network structures, which make the number of parameters and inference time increase exponentially. This paper introduces Res2Net in target detection task to speaker recognition task, and verifies its effectiveness and robustness in speaker recognition task. And FullRes2Net is improved and proposed to have stronger multi-scale feature extraction capability without increasing the number of parameters, and 17% performance improvement compared to Res2Net. Meanwhile, in order to solve the problems of existing attention methods improve the shortcomings of convolution itself and further enhance the feature extraction ability of convolutional neural networks, mixed time-frequency channel attention is proposed. Experiments are conducted on the Voxceleb dataset, and the results show that the proposed method effectively improves the feature extraction ability and generalization ability of the system, with a 34% performance improvement compared to Res2Net, and outperforms advanced speaker recognition systems using complex structures, which is an end-to-end structure with fewer parameters and higher efficiency, suitable for applications in realistic scenarios.

Key words: speaker recognition, end-to-end, attention mechanisms

摘要： 说话人识别系统中轻量卷积神经网络的特征提取能力弱、识别效果差。而为了提升特征提取能力，许多方法使用了更深、更宽、更复杂的网络结构，使得参数量和推理时间成倍增加。将目标检测任务中的轻量网络Res2Net引入到说话人识别任务中，验证了它在说话人识别任务中的有效性和鲁棒性。并改进提出了FullRes2Net，它拥有更多、更大的感受野组合。在几乎没有增加参数量的情况下，相比于Res2Net，性能提升了17%。同时，为了解决现有注意力方法存在的问题改善卷积本身的缺点，进一步提升卷积神经网络的特征提取能力，提出了混合时频通道注意力。它可以对音频特征的时间、频率、通道维度进行交互，捕捉特征间的依赖，从而有效增强卷积神经网络的特征提取能力。在Voxceleb数据集上进行了实验，结果表明提出的方法有效地提升了系统的特征提取能力和泛化能力，相较于Res2Net性能提升了34%，并优于使用复杂结构的先进说话人识别系统，是一种参数量更少、效率更高的端到端结构，适合在现实场景中的应用。

关键词: 说话人识别, 端到端, 注意力机制

DENG Lihong, DENG Fei, ZHANG Gexiang, YANG Qiang. Multi-Scale End-to-End Speaker Recognition System Based on Improved Res2Net[J]. Computer Engineering and Applications, 2023, 59(24): 110-120.

邓力洪, 邓飞, 张葛祥, 杨强. 改进Res2Net的多尺度端到端说话人识别系统[J]. 计算机工程与应用, 2023, 59(24): 110-120.

References

[1] HANSEN J，HASAN T.Speaker recognition by machines and humans：a tutorial review[J].IEEE Signal Processing Magazine，2015，32（6）：74-99.
[2] STOLL L L.Finding difficult speakers in automatic speaker recognition[D].Berkeley：University of California，2011.
[3] MATEJKA P，GLEMBEK O，CASTALDO F，et al.Full-covariance UBM and heavy tailed PLDA in i-vector speaker verification[C]//Proceedings of the 2011 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2011：4828-4831.
[4] CUMANI S，PLCHOT O，LAFACE P.Probabilistic linear discriminant analysis of i-vector posterior distributions[C]//Proceedings of 2013 IEEE International Conference on Acoustics，Speech and Signal Processing，Vancouver，BC，Canada，2013：7644-7648.
[5] MCLAREN M，FERRER L，CASTAN D，et al.The speakers in the wild （SITW） speaker recognition database[C]//Proceedings of the Interspeech，2016：818-822.
[6] NAGRANI A，CHUNG J S，ZISSERMAN A.VoxCeleb：a large-scale speaker identification dataset[C]//Proceedings of the Interspeech，2017：2616-2620.
[7] HE K，ZHANG X，REN S，et al.Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition（CVPR），Las Vegas，NV，USA，2016：770-778.
[8] ZEINALI H，WANG S，SILNOVA A，et al.BUT system description to VoxCeleb speaker recognition challenge 2019[J].arXiv：1910.12592，2019.
[9] JUNG J W，KIM S B，SHIM H J，et al.Improved RawNet with feature map scaling for text-independent speaker verification using raw waveforms[C]//Proceedings of the Interspeech，2020：1496-1500.
[10] WANG R，AO J，ZHOU L，et al.Multi-view self-attention based transformer for speaker recognition[C]//Proceedings of 2022 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2022：6732-6736.
[11] ZHOU J F，JIANG T，LI Z，et al.Deep speaker embedding extractionwith channel-wise feature responses and additive super vision softmax loss function[C]//Proceedings of the Interspeech，2019：2883-2887.
[12] HU J，SHEN L，SUN G.Squeeze-and excitation networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition，2018：7132-7141.
[13] WOO S，PARK J，LEE J Y，et al.Cbam：convolutional block attention module[C]//Proceedings of the 2018 European Conference on Computer Vision，2018：3-19.
[14] YADAV S，RAI A K.Frequency and temporal convolutional attention for text-independent speaker recognition[C]//Proceedings of the 2020 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2020：6794-6798.
[15] GAO S，CHENG M，ZHAO K，et al.Res2Net：a new multi-scale backbone architecture[J].IEEE Transactions on Pattern Analysis and Machine Intelligence，2021，43：652-662.
[16] WANG H，FAN Y，WANG Z，et al.Parameter-free spatial attention network for person re-identification[J].arXiv：1811.12150，2018.
[17] WANG Q，WU B，ZHU P，et al.ECA-Net：efficient channel attention for deep convolutional neural networks[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR），2020：11531-11539.
[18] NAGRANI A，CHUNG J S，XIE W D，et al.Voxceleb：large-scale speaker verification in the wild[J].Computer Speech & Language，2020（60）：101027.
[19] CHUNG J S，NAGRANI A，ZISSERMAN A.VoxCeleb2：deep speaker recognition[C]//Proceedings of the Interspeech，2018：1086-1090.
[20] XIE W，NAGRANI A，CHUNG J S，et al.Utterance-level aggregation for speaker recognition in the wild[C]//Proceedings of the 2019 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2019：5791-5795.
[21] AMIRHOSSEIN H，ETEMAD A.A deep neural network for short-segment speaker recognition[J].arXiv：1907.
10420，2019.
[22] KINGMA D P，BA J.Adam：a method for stochastic optimization[J].arXiv：1412.6980，2014.
[23] WAN L，WANG Q，PAPIR A，et al.Generalized end-to-end loss for speaker verification[C]//Proceedings of the 2018 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP），2018：4879-4883.
[24] MARTIN A F，GREENBERG C S.The NIST 2010 speaker recognition evaluation[C]//Proceedings of the Interspeech，2010.
[25] ZHU G，JIANG F，DUAN Z.Y-Vector：multiscale waveform encoder for speaker embedding[C]//Proceedings of the Interspeech，2021：96-100.
[26] VAN DER MAATEN L，HINTON G E.Visualizing data using T-SNE[J].Journal of Machine Learning Research，2008（9）：2579-2605.