融合模型量化和缓存优化的实时语音监测方法

doi:10.3778/j.issn.1002-8331.2406-0033

摘要/Abstract

摘要： 针对文化市场新业态的监管需求，提出一种融合模型量化和缓存优化的实时语音监测方法。通过模型量化，在有限精度损失的情况下优化大模型加载速度并降低系统资源开销。在数据缓存优化方面采用最长公共前缀匹配策略动态调整缓冲区设置，提升语音转录内容上下文关联，同时降低词错率（word error rate，WER）。针对敏感内容训练基于BERT-TextCNN的敏感信息检测模型，建立非现场监管语音监测体系，实现对演出内容的实时监测和预警。实验结果表明，提出的模型量化方法在Whisper-large-v3预训练模型的FP16和FP32两个基准测试中分别能够提升2.62倍和2.11倍推理速度，与现有方法相比具有优势；在语音识别准确率和延迟方面，采用缓存优化策略后语音转录延迟平均降低了12.88%，中文词错率降低了14.42%；在语言类演出节目构成的真实数据集上进行实验，BERT-TextCNN模型对敏感内容的检测准确率达到92.66%，与其他方法相比具有更高的精确度和召回率，证明了所提方法能够有效支撑对小剧场等文化演出形式的非现场监管。

关键词: 语音识别, 模型量化, 最长公共前缀, 敏感内容检测

Abstract: A real-time streaming transcription monitoring method that integrates model quantification and buffer optimization is proposed to meet the regulatory needs of new business formats in the cultural market. By quantifying the model, the inference speed of large models is optimized and the system resource overhead is reduced in the case of limited accuracy loss. In terms of data buffering optimization, the longest common prefix matching strategy is used to dynamically adjust the buffer capacity to improve the contextual relevance of speech transcription content and reduce the word error rate (WER). A BERT-TextCNN sensitive information detection model is trained for sensitive content, and an off-site supervision voice monitoring system is constructed to achieve detection and early warning of cultural performance content. Experimental results show that the proposed model quantization method can increase the inference speed by 2.62 times and 2.11 times respectively in the two benchmark tests of FP16 and FP32 of the Whisper-large-v3 pre-training model, which has advantages over existing methods. In terms of speech recognition, the average transcription delay of the original model is reduced by 12.88%, and the Chinese word error rate is reduced by 14.42%. Experiments are conducted on real datasets composed of language performances, and the detection accuracy of sensitive content reaches 92.66%, which has higher precision and recall than other methods and demonstrates that the proposed method can effectively support off-site supervision of performances such as live streaming and little theatre.

Key words: speech recognition, model quantization, longest common prefix, sensitive content detection

吴非, 沈润楠, 陈宇. 融合模型量化和缓存优化的实时语音监测方法[J]. 计算机工程与应用, 2025, 61(16): 215-223.

WU Fei, SHEN Runnan, CHEN Yu. Model Quantization and Buffer Optimization Based Real-Time Streaming Transcription Monitoring Method[J]. Computer Engineering and Applications, 2025, 61(16): 215-223.

参考文献

[1] BAEVSKI A, ZHOU H, MOHAMED A, et al. wav2vec 2.0: a framework for self-supervised learning of speech representations s[C]//Advances in Neural Information Processing Systems 33, 2020: 12449-12460.
[2] HIGUCHI Y, OGAWA T, KOBAYASHI T, et al. BECTRA: transducer-based end-to-end ASR with BERT-enhanced encoder[C]//Proceedings of the 2023 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2023: 1-5.
[3] TIAN J C, YU J W, WENG C, et al. Improving mandarin end-to-end speech recognition with word N-gram language model[J]. IEEE Signal Processing Letters, 2022, 29: 812-816.
[4] XUE J, WANG P D, LI J Y, et al. Large-scale streaming end-to-end speech translation with neural transducers[J]. arXiv: 2204.05352, 2022.
[5] HAN M L, DONG L H, LIANG Z L, et al. Improving end-to-end contextual speech recognition with fine-grained contextual knowledge selection[C]//Proceedings of the 2022 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2022: 8532-8536.
[6] RADFORD A, KIM J W, XU T, et al. Robust speech recognition via large-scale weak supervision[C]//Proceedings of the 40th International Conference on Machine Learning, 2023: 28492-28518.
[7] LIU D N, SPANAKIS G, NIEHUES J. Low-latency sequence-to-sequence speech recognition and translation by partial hypothesis selection[J]. arXiv:2005.11185, 2020.
[8] GULATI A, QIN J, CHIU C C, et al. Conformer: convolution-augmented transformer for speech recognition[J]. arXiv:2005. 08100, 2020.
[9] HE Y Z, SAINATH T N, PRABHAVALKAR R, et al. Streaming end-to-end speech recognition for mobile devices[C]//Proceedings of the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2019: 6381-6385.
[10] LIU A H, HSU W N, AULI M, et al. Towards end-to-end unsupervised speech recognition[C]//Proceedings of the 2022 IEEE Spoken Language Technology Workshop. Piscataway: IEEE, 2023: 221-228.
[11] CHAN W, PARK D, LEE C, et al. SpeechStew: simply mix all available speech recognition data to train one large neural network[J]. arXiv:2104.02133, 2021.
[12] KIM K, WU F, PENG Y F, et al. E-branchformer: branchformer with enhanced merging for speech recognition[C]//Proceedings of the 2022 IEEE Spoken Language Technology Workshop. Piscataway: IEEE, 2023: 84-91.
[13] 沈逸文, 孙俊. 结合Transformer的轻量化中文语音识别[J]. 计算机应用研究, 2023, 40(2): 424-429.
SHEN Y W, SUN J. Lightweight Chinese speech recognition with transformer[J]. Application Research of Computers, 2023, 40(2): 424-429.
[14] 张瑞珍, 韩跃平, 张晓通. 基于深度LSTM的端到端的语音识别[J]. 中北大学学报(自然科学版), 2020, 41(3): 244-248.
ZHANG R Z, HAN Y P, ZHANG X T. End-to-end speech recognition based on depth-gated LSTM[J]. Journal of North University of China (Natural Science Edition), 2020, 41(3): 244-248.
[15] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30, 2017.
[16] VANHOLDER H. Efficient inference with TensorRT[C]//GPU Technology Conference, 2016: 1-24.
[17] FANG J R, YU Y, ZHAO C D, et al. TurboTransformers: an efficient GPU serving system for transformer models[C]//Proceedings of the 26th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming. New York: ACM, 2021: 389-402.
[18] WANG X H, XIONG Y, WEI Y, et al. LightSeq: a high performance inference library for transformers[J]. arXiv:2010. 13887, 2020.
[19] ASUDANI D S, NAGWANI N K, SINGH P. Impact of word embedding models on text analytics in deep learning environment: a review[J]. Artificial Intelligence Review, 2023, 56(9): 10345-10425.
[20] SELVA BIRUNDA S, KANNIGA DEVI R. A review on word embedding techniques for text classification[C]//Proceedings of the 2020 International Conference on Innovative Data Communication Technologies and Application. Singapore: Springer, 2021: 267-281.
[21] 黄诚, 赵倩锐. 基于语言模型词嵌入和注意力机制的敏感信息检测方法[J]. 计算机应用, 2022, 42(7): 2009-2014.
HUANG C, ZHAO Q R. Sensitive information detection method based on attention mechanism-based ELMo[J]. Journal of Computer Applications, 2022, 42(7): 2009-2014.
[22] SHARMIN S, CHAKMA D. Attention-based convolutional neural network for Bangla sentiment analysis[J]. AI & Society, 2021, 36(1): 381-396.
[23] LIU Y, YANG C Y, YANG J. A graph convolutional network-based sensitive information detection algorithm[J]. Complexity, 2021(1): 6631768.
[24] 张泽锋, 毛存礼, 余正涛, 等. 融入领域术语词典的司法舆情敏感信息识别[J]. 中文信息学报, 2022, 36(9): 76-83.
ZHANG Z F, MAO C L, YU Z T, et al. Sensitive judicial public opinion information recognition with the domain terminology dictionary[J]. Journal of Chinese Information Processing, 2022, 36(9): 76-83.
[25] 金秋, 林馥, 裴斐. 基于层次聚类的敏感信息安全过滤模型研究[J]. 计算机仿真, 2023, 40(10): 296-299.
JIN Q, LIN F, PEI F. Research on security filtering model of sensitive information based on hierarchical clustering[J]. Computer Simulation, 2023, 40(10): 296-299.
[26] 刘聪, 王永利, 周子韬, 等. 结合触发事件及词性分析的敏感信息识别方法[J]. 计算机工程与应用, 2020, 56(20): 132-137.
LIU C, WANG Y L, ZHOU Z T, et al. Sensitive information recognition method combining trigger event and part of speech analysis[J]. Computer Engineering and Applications, 2020, 56(20): 132-137.