Backtracking matching Chinese segmentation method

doi:10.3778/j.issn.1002-8331.2009.22.043

Computer Engineering and Applications ›› 2009, Vol. 45 ›› Issue (22): 132-134.DOI: 10.3778/j.issn.1002-8331.2009.22.043

• 数据库、信息处理 • Previous Articles Next Articles

Backtracking matching Chinese segmentation method

ZHANG Jin-song，YUAN Jian

College of Computing & Electrical Engineering，University of Shanghai for Science and Technology，Shanghai 200093，China

Received:2008-05-15 Revised:2008-08-14 Online:2009-08-01 Published:2009-08-01
Contact: ZHANG Jin-song

回溯正向匹配中文分词算法

张劲松，袁健

上海理工大学计算机与电气工程学院，上海 200093

通讯作者: 张劲松

Abstract

Abstract: Chinese word segmentation methods of the backtracking matching method is proposed based on the maximum matching method.The text is pretreatment by the BM in the first，then，the text is cut into shorter lengths granular text；Ambiguity field is found effective by forward matching method，backtracking matching method and last words matching Method；Crossing ambiguity field is cut by Long term priorities and 2-words rules.In the last，the algorithm of the qualitative analysis shows that the method of the advanced nature.The large number of randomly selected language materials being tested and results show that the method is effective.

Key words: Chinese word segmentation, Maximum Matching（MM） methods, backtracking matching, crossing ambiguity

摘要： 在最大匹配法（The Maximum Matching Method）的基础上，提出了一种回溯正向匹配（The Backtracking Matching Method）的中文分词方法。该方法首先对待切文本进行预处理，将文本分割成长度较短的细粒度文本；利用正向匹配、回溯匹配和尾词匹配来有效发现歧义字段；利用长词优先兼顾2词簇的方式对交集型歧义字段进行切分。最后对该算法进行的定性分析说明了该方法的先进性，从随机抽取大量语料实验结果上证明了该方法的有效性。

关键词: 中文分词, 最大匹配法, 回溯匹配, 交集型歧义

ZHANG Jin-song，YUAN Jian. Backtracking matching Chinese segmentation method[J]. Computer Engineering and Applications, 2009, 45(22): 132-134.

张劲松，袁健. 回溯正向匹配中文分词算法[J]. 计算机工程与应用, 2009, 45(22): 132-134.

[1]	TU Wenbo, YUAN Zhenming, YU Kai. Convolutional Neural Networks Without Pooling Layer for Chinese Word Segmentation [J]. Computer Engineering and Applications, 2020, 56(2): 120-126.
[2]	SUN Baoshan, LI Wei. Recurrent Neural Network for Chinese Word Segmentation with Peephole-Connections [J]. Computer Engineering and Applications, 2019, 55(19): 160-165.
[3]	CHENG Yusi1, SHI Yuntao2. Domain specific Chinese word segmentation [J]. Computer Engineering and Applications, 2018, 54(17): 30-34.
[4]	ZHAO Weifeng1，2, ZHANG Qin1. Automatic identification of address description in unstructured Chinese natural language [J]. Computer Engineering and Applications, 2016, 52(23): 19-24.
[5]	ZHU Yanhui, LIU Jing, XU Yeqiang, TIAN Hailong, MA Jin. Chinese word segmentation research based on Conditional Random Field [J]. Computer Engineering and Applications, 2016, 52(15): 97-100.
[6]	ZHANG Sifa, MA Yongge. Design and implementation of vertical search engine for field of geosciences [J]. Computer Engineering and Applications, 2012, 48(33): 85-88.
[7]	YE Jiping, ZHANG Guizhu. Research and improvement of Chinese word segmentation dictionary [J]. Computer Engineering and Applications, 2012, 48(23): 139-142.
[8]	ZHAO Youqiao1, ZHANG Shanshan1, LU Songfeng1, WU Zhijie2. COX：Chinese-oriented XML compressor with high compression ratio [J]. Computer Engineering and Applications, 2012, 48(17): 143-147.
[9]	LI Guohe1，2，3, LIU Guangsheng1，2，3, QIN Bobo1，2，3, WU Weijiang1，2，3, LI Hongqi1，2，3. Method of Chinese word rough segmentation by maximum match and ambiguity detection algorithms [J]. Computer Engineering and Applications, 2012, 48(14): 139-142.
[10]	YU Jiangde¹，WANG Xijie¹，FAN Xiaozhong². Comparing of importance of above-context versus below-context for Chinese word segmentation [J]. Computer Engineering and Applications, 2011, 47(4): 117-120.
[11]	LIU Ronghui^1，2，ZHENG Jianguo¹. Clustering algorithm in Deep Web based on Chinese word segmentation [J]. Computer Engineering and Applications, 2011, 47(4): 138-140.
[12]	YOU Huili，YAN Li，YANG Xiaodong. Research on combinational ambiguity strings in Chinese word segmentation [J]. Computer Engineering and Applications, 2011, 47(31): 125-127.
[13]	ZHANG Qing-yang,CHAI Sheng. Chinese word segmentation dictionary using two-level index [J]. Computer Engineering and Applications, 2009, 45(19): 139-141.
[14]	LIU Dan,FANG Wei-guo,ZHOU Hong. Performance of smoothing algorithm in Chinese word segmentation by bigram [J]. Computer Engineering and Applications, 2009, 45(17): 33-36.
[15]	ZHOU Sheng^1,2,HU Xiao-feng¹,LUO Pi¹. Research on automatization of virtual news system [J]. Computer Engineering and Applications, 2008, 44(36): 20-23.

Backtracking matching Chinese segmentation method

回溯正向匹配中文分词算法

PDF

Knowledge

Abstract

Cite this article

share this article

References

Related Articles 15

Recommended Articles

Metrics