Logo image
Chinese Spell Checking Based on Noisy Channel Model
Thesis

Chinese Spell Checking Based on Noisy Channel Model

Chiu, Hsun-Wen
Masters, 國立清華大學, 資訊系統與應用研究所
2013

Abstract

雜訊通道模型 語言模型 網路語料 混淆字集 Noisy Channel Model Character-based Language Model Web Corpus Confusion Set
Chinese spell checking is an important component of many Chinese NLP applications, including word processors, search engines, and automatic essay rating. Compared to English, Chinese has no word boundaries, and there are various Chinese input methods that cause different kinds of typos. Therefore, it is more difficult to develop a spell checker for Chinese. In this paper, we introduce a novel method for correcting Chinese errors based on sound or shape similarity. In our approach, potential typos in a given sentence are then corrected using a channel model and a character-based language model in the noisy channel model. In the training phase, we estimate the channel probabilities for each character based on ngrams in Web corpus. At run-time, the system generates correction candidates for each character in the given sentence and selects the appropriate correction using the channel model and the language model. The experimental results show that the proposed method achieves significantly better accuracy and recall than more complicated methods in the previous work.

Metrics

1 Record Views

Details

Logo image