Review of Research on Cross-Lingual Summarization

doi:10.3778/j.issn.1002-8331.2208-0035

Abstract

Abstract: With the development of the Internet, articles in various languages spring up, in order for users to quickly understand the main content of the article, it is necessary to obtain the main text information between different languages. Cross-lingual summarization is a method that can reflect the main idea of text extracted from multiple languages by computer, which can effectively solve the above problems. From the perspective of the development of the cross-lingual summarization method, this paper firstly makes a comprehensive survey on the research work of the cross-lingual summarization method, and combs the development process of the cross-lingual summarization method. Secondly, the key technologies are discussed and analyzed, the differences and shortcomings of these methods are summarized, and the main line of research from the initial pipeline-based method to the end-to-end method emerged after the popularization of deep learning is outlined. Finally, the challenges and future research trends of cross-lingual summarization are analyzed and prospected.

Key words: cross-lingual summarization, deep learning, pipeline, end-to-end method, natural language processing

摘要： 随着互联网的发展，各种语言文章涌现。为了用户能快速地了解文章的主要内容，需要获取不同语言之间的文本主旨信息。跨语言摘要是利用计算机从多种语言文本中提炼出一种能反映文本主旨的方法。从跨语言摘要方法发展的角度切入，对跨语言摘要的研究工作进行了全面的调查，梳理了跨语言摘要方法的发展过程，定义了跨语言摘要的任务。对其关键技术进行讨论与分析，总结出这些方法的差异与不足，勾画出从最初的基于管道式方法到深度学习普及后出现的基于端到端的方法的研究主线，并对跨语言摘要数据集进行了分析总结。最后对跨语言摘要面临的挑战和未来研究趋势进行分析与展望。

关键词: 跨语言摘要, 深度学习, 管道式, 端到端方法, 自然语言处理

ZHENG Bofei, YUN Jing, LIU Limin, JIAO Lei, YUAN Jingshu. Review of Research on Cross-Lingual Summarization[J]. Computer Engineering and Applications, 2023, 59(13): 49-60.

郑博飞, 云静, 刘利民, 焦磊, 袁静姝. 跨语言摘要方法研究综述[J]. 计算机工程与应用, 2023, 59(13): 49-60.

References

[1] LEUSKI A，LIN C Y，ZHOU L，et al.Cross-lingual C* ST* RD：English access to Hindi information[J].ACM Transactions on Asian Language Information Processing（TALIP），2003，2（3）：245-269.
[2] ORǎSAN C，CHIOREAN O A.Evaluation of a cross-lingual romanian-english multi-document summariser[C]//Proceedings of the 6th International Conference on Language Resources and Evaluation，Marrakech，Morocco，2008.
[3] WAN X J.Using bilingual information for cross-language document summarization[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics：Human Language Technologies，2011：1546-1555.
[4] YAO J，WAN X，XIAO J.Phrase-based compressive cross-language summarization[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing，2015：118-127.
[5] ZHANG J，ZHOU Y，ZONG C.Abstractive cross-language summarization via translation model enhanced predicate argument structure fusing[J].IEEE/ACM Transactions on Audio，Speech，and Language Processing，2016，24（10）：1842-1853.
[6] WAN X J，LUO F L，SUN X，et al.Cross-language document summarization via extraction and ranking of multiple summaries[J].Knowledge and Information Systems，2019，58（2）：481-499.
[7] WAN X J，LI H Y，XIAO J G.Cross-language document summarization based on machine translation quality prediction[C]//Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics，2010：917-926.
[8] BAI Y，HUANG H，FAN K，et al.Unifying cross-lingual summarization and machine translation with compression rate[C]//Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval，2022：1087-1097.
[9] NALLAPATI R，ZHOU B，GULCEHRE C，et al.Abstractive text summarization using sequence-to-sequence RNNs and beyond[J].arXiv：1602.06023，2016.
[10] BING L，LI P，LIAO Y，et al.Abstractive multi-document summarization via phrase selection and merging[J].arXiv：1506.01597，2015.
[11] GILDEA D.JURAFSKY D.Automatic labeling of semantic roles[J].Computational Linguistics，2002，28（3）：245-288.
[12] LINHARES PONTES E，HUET S，TORRES-MORENO J M，et al.Cross-language text summarization using sentence and multi-sentence compression[C]//Proceedings of the International Conference on Applications of Natural Language to Information Systems，2018：467-479.
[13] ANI N.KATHLEEN M.A survey of text summarization techniques[M]//Mining text data.Boston，MA：Springer， 2012：43-76.
[14] 殷明明.跨语言的句子摘要研究[D].苏州：苏州大学，2020.
DUAN M M.Cross-language sentence summarization research[D].Suzhou：Suzhou University，2020.
[15] CAO Y，LIU H，WAN X.Jointly learning to align and summarize for neural cross-lingual summarization[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：6220-6231.
[16] BAI Y，GAO Y，HUANG H.Cross-lingual abstractive summarization with limited parallel resources[J].arXiv：2105.13648，2021.
[17] ZHU J N，ZHOU Y，ZHANG J N，et al.Attend translate and summarize：an efficient method for neural cross-lingual summarization[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics，2020：1309-1321.
[18] JIANG S，TU D，CHEN X，et al.CptGraphSum：let key clues guide the cross-lingual abstractive summarization[J].arXiv：2203.02797，2022.
[19] 王剑，张莹，余正涛，等.使用词对齐半监督对抗学习的汉越跨语言摘要生成方法[J].小型微型计算机系统，2022，43（5）：992-997.
WANG J，ZHANG Y，YU Z T，et al.Semi-supervised adversarial Chinese-vietnamese cross-lingual summarization generation method using word alignment[J].Journal of Chinese Computer Systems，2022，43（5）：992-997.
[20] DOU Z Y，KUMAR S，TSVERKOV Y.A deep reinforced model for zero-shot cross-lingual summarization with bilingual semantic similarity rewards[J].arXiv：2006. 15454，2020.
[21] AYANA，SHEN S Q，CHEN Y，et al.Zero-shot cross-lingual neural headline generation[J].IEEE/ACM Transactions on Audio，Speech，and Language Processing，2018，26（12）：2319-2327.
[22] DUAN X，YIN M，ZHANG M，et al.Zero-shot cross-lingual abstractive sentence summarization through teaching generation and attention[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics，2019：3162-3172.
[23] VASWANI A，SHAZEER N，PARMAR N，et al.Attention is all you need[C]//Advances in Neural Information Processing Systems，2017.
[24] NGUYEN T，TUAN L A.Improving neural cross-lingual summarization via employing optimal transport distance for knowledge distillation[J].arXiv：2112.03473，2021.
[25] FEYDY J，SéJOURNé T，VIALARD F X，et al.Interpolating between optimal transport and mmd using sinkhorn divergences[C]//Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics，2019：2681-2690.
[26] ZHU J N，WANG Q，WANG Y N，et al.NCLS：neural cross-lingual summarization[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing（EMNLP-IJCNLP），2019：3054-3064.
[27] TAKASE S，OKAZAKI N.Multi-task learning for cross-lingual abstractive summarization[J].arXiv：2010.07503，2020.
[28] ZHU J N，LI H，LIU T，et al.MSMO：multimodal summarization with multimodal output[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing，2018：4154-4164.
[29] MELVIN J，SCHUSTER M，CHEN Z F.et al.Google’s multilingual neural machine translation system：enabling zero-shot translation[J].Transactions of the Association for Computational Linguistics，2017，5：339-351.
[30] TASKASE S，OKAZAKI N.Positional encoding to control output sequence length[J].arXiv：1904.07418，2019.
[31] LIANG Y，MENG F，ZHOU C，et al.A variational hierarchical model for neural cross-lingual summarization[J].arXiv：2203.03820，2022.
[32] SOHN K，LEE H，YAN X.Learning structured output representation using deep conditional generative models[C]//Advances in Neural Information Processing Systems，2015.
[33] DYER C，CHAHUNNNEAU V，SMITH N A.A simple，fast，and effective reparameterization of IBM model 2[C]//Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，2013：644-648.
[34] MIHALCEA R，TARAU P.Textrank：bringing order into text[C]//Proceedings of the 2004 Conference on empirical Methods in Natural Language Processing，2004：404-411.
[35] BABAR S A，PATIL P D.Improving performance of text summarization[J].Procedia Computer Science，2015，46：354-363.
[36] NETO J L，FREITAS A A，KAESTNER C A A.Automatic text summarization using a machine learning approach[C]//Proceedings of Brazilian Symposium on Artificial Intelligence.Berlin，Heidelberg：Springer，2002：205-215.
[37] NOMOTO T，MATSUMOTO Y.A new approach to unsupervised text summarization[C]//Proceedings of the 24th Annual International ACM SIGIR Conference on Research And Development in information Retrieval，2001：26-34.
[38] TAS O，KIYANI F.A survey automatic text summarization[J].Press Academia Procedia，2007，5（1）：205-213.
[39] CHEN X L，HASSAN A，HASSAN N，et al.Multi-source cross-lingual model transfer：learning what to share[J].arXiv.1810.03552，2018.
[40] SCHUSTER T，RAM O，BARZILAY R，et al.Cross-lingual alignment of contextual word embeddings，with applications to zero-shot dependency parsing[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics：Human Language Technologies，Volume 1（Long and Short Papers），2019：1599-1613.
[41] QIU X，SUN T，XU Y，et al.Pre-trained models for natural language processing：a survey[J].Science China Technological Sciences，2020，63（10）：1872-1897.
[42] XU R，ZHU C，SHI Y，et al.Mixed-lingual pre-training for cross-lingual summarization[J].arXiv：2010.08892，2020.
[43] WANG J，MENG F，LU Z，et al.ClidSum：a benchmark dataset for cross-lingual dialogue summarization[J].arXiv：2202.05599，2022.
[44] MA S，DONG L，HUANG S，et al.DeltaLM：encoder-decoder pre-training for language generation and translation by augmenting pretrained multilingual encoders[J].arXiv：2106.13736，2021.
[45] CHI Z，DONG L，MA S，et al.mT6：multilingual pretrained text-to-text transformer with translation pairs[J].arXiv：2104.08692，2021.
[46] LIU Y，GU J，GOYAL N，et al.Multilingual denoising pre-training for neural machine translation[J].Transactions of the Association for Computational Linguistics，2020，8：726-742.
[47] XUE L，CONSTANT N，Roberts A，et al.mT5：a massively multilingual pre-trained text-to-text transformer[J].arXiv：2010.11934，2020.
[48] CHI Z，DONG L，WEI F，et al.InfoXLM：an information-theoretic framework for cross-lingual language model pre-training[J].arXiv：2007.07834，2020.
[49] WIETING J，GIMPEL K，NEUBIG G，et al.Simple and effective paraphrastic similarity from parallel translations[J].arXiv：1909.13872，2019.
[50] WU Y H，SCHUSTER M，CHEN Z F，et al.Google’s neural machine translation system：bridging the gap between human and machine translation[J].arXiv：1609. 08144，2016.
[51] HERMANN K M，KOCISKY T，GREFENSTETTE E，et al.Teaching machines to read and comprehend[C]//Advances in Neural Information Processing Systems，2015，28：1693-1701.
[52] SEE A，LIU P J，MANNINH C D.Get to the point：Summarization with pointer-generator networks[J].arXiv：1704.04368，2017.
[53] HU B，CHEN Q，ZHU F.LCSTS：a large scale Chinese short text summarization dataset[J].arXiv：1506.05865，2015.
[54] 李金鹏，张闯，陈小军，等.自动文本摘要研究综述[J].计算机研究与发展，2021，58（1）：1-21.
LI J P，ZHANG C，CHEN X J，et al.Survey on automatic text summarization[J].Journal of Computer Research and Development，2021，58（1）：1-21.
[55] HUANG S D，GRAFF D.Chinese-english translation lexicon version 3.0 LDC2002L27[DB/OL].Philadelphia：Linguistic Data Consortium（2002-06-17）[2022-10-12].https：//catalog.ldc.upenn.edu/LDC2002L27.
[56] HUANG S D，GRAFF D，DODDINGTON G.Multiple-translation Chinese corpus LDC2002T01[DB/OL].Philadelphia：Linguistic Data Consortium（2002-02-04）[2022-10-12].https：//catalog.ldc.upenn.edu/LDC2002T01.
[57] LIU P，YUAN W，FU J，et al.Pre-train，prompt，and predict：A systematic survey of prompting methods in natural language processing[J].arXiv：2107.13586，2021.
[58] HASAN T，BHATTACHARJEE A，AHMAD W U，et al.CrossSum：beyond English-centric cross-lingual abstractive text summarization for 1500+ language Pairs[J].arXiv：2112.08804，2021.
[59] LIN C Y.ROUGE：a package for automatic evaluation of summaries[C]//Proceedings of the Workshop on Text Summarization Branches Out（WAS 2004），2004：74-81.