关键词: Ajax爬虫, 带权状态融合树, 文本挖掘, 文本特征树

Abstract: Ajax technology has been widely applied in deep web application development. This paper constructs a Weighted State Fusion Tree （WSFT） model to pre-process the text information in web page with Ajax technology which has multiple states with strong correlation. Firstly, the current approach of Ajax page data collection is optimized by regarding text feature tree as a fingerprint to traverse through the multiple states. Secondly, the transition weight with StatusRank method is calculated for each states of the Ajax page. The state transition information is analyzed. Finally, a WSFT is generated. The experimental results show that the proposed method can effectively obtain the text information in Ajax page with multiple states, and help the follow-up important text extraction of web mining.

Key words: Ajax crawler, weighted state fusion tree, text mining, text feature tree