Logo image
Syllable Word Segmentation for Mandarin Chinese via Double Ranking of the Left and Right Context
Dissertation

Syllable Word Segmentation for Mandarin Chinese via Double Ranking of the Left and Right Context

Jiang, Tian-Jian
Doctor of Philosophy (PHD), 國立清華大學, 資訊工程學系
2011

Abstract

中文斷詞 交疊歧義 雙重排名 Chinese word segmentation overlapping ambiguity double rank
Syllable word segmentations as a part of Chinese phonetic input methods (CPIM) involve more overlapping boundaries than word segmentations because of homophone ambiguities. A CPIM usually assumes that the input is a complete sentence, and evaluates the performance based on a well-formed corpus. However, most Pinyin users prefer progressive text entry in short chunks, mainly in one or two words each, which is even more popular on handheld devices with limited computing power. Short chunks do not provide enough contexts to perform the best possible syllable-to-character conversion, especially when a chunk consists of overlapping boundaries. Those overlapping ambiguities show directional tendencies. This dissertation proposes a double ranking (DR) strategy on the left and right context. Experiments show that DR has the benefits of less memory with competitive performance compared to the frequency-based method (low memory and fast) and the conditional random fields model (larger memory and slower).

Metrics

1 Record Views

Details

Logo image