计算机工程与应用 ›› 2015, Vol. 51 ›› Issue (16): 130-135.

• 数据库、数据挖掘、机器学习 • 上一篇    下一篇

基于关键词的维吾尔单文档自动文摘技术研究

买哈铺热提·外力1,赵梦原2,艾斯卡尔·艾木都拉1   

  1. 1.新疆大学 信息科学与工程学院,乌鲁木齐 830046
    2.清华大学 语音和语言技术研究中心,北京 100086
  • 出版日期:2015-08-15 发布日期:2015-08-14

Keyword based Uyghur single document summarization

Mahpirat Wali1, ZHAO Mengyuan2, Askar Hamdulla1   

  1. 1.Institute of Information Science and Engineering, Xinjiang University, Urumqi 830046, China
    2.Research Center of Speech and Language Technology, Tsinghua University, Beijing 100086, China
  • Online:2015-08-15 Published:2015-08-14

摘要: 以互联网为代表的信息技术的发展使人们索取信息变得前所未有的便捷,同时也对如何有效利用信息提出了挑战。自动文摘技术通过自动选择文档中的代表句子,可以极大提高信息使用的效率。近年来,基于英文和中文的自动文摘技术获得广泛关注并取得长足进展,而对少数民族语言的自动文摘研究还不够充分,例如维吾尔语。构造了一个面向维吾尔语的自动文摘系统。首先利用维吾尔语的语言学知识对文档进行预处理,之后对文档进行了关键词提取,利用这些关键词进行了抽取式自动文摘。比较了基于TF-IDF和基于TextRank的两种关键词提取算法,证明TextRank方法提取出的关键词更适合自动文摘应用。通过研究证明了在充分考虑到维吾尔语语言信息的前提下,基于关键词的自动文摘方法可以取得让人满意的效果。

关键词: 维吾尔文, 自动文摘, TF-IDF算法, Textrank, ROUGE

Abstract: As represented by the Internet, development of information technology has enabled people to obtain information easier than ever before, but it also presents challenges to the effective use of information. Automatic summarization techniques greatly improve efficiency in the use of information by automatically selecting representatives of the sentences in the document. In recent years, automatic summarization techniques based on English and Chinese received wide attention and achieved significant progress while the automatic summarization of minority languages is not sufficient, such as Uyghur language. This paper constructs a Uyghur-oriented automatic summarization system. Uyghur linguistic knowledge is used to handle the document, and then keywords which are extracted from the document is used for automatic text summarization. Two different TF-IDF-based and TextRank-based extraction algorithms are compared; it proves TextRank method is more suitable for automatic text summarization. It is demonstrated that on the premise of full account of Uygur language information, automatic text summarization based on keywords can achieve satisfactory results.

Key words: Uyghur, automatic summarization, TF-IDF algorithm, TextRank, ROUGE