Abstract
We present a vision-based sign language recognition system which works efficiently to recognize the Taiwan Sign Language. Sign language can be divided into a sequence of sign-words, and sign-word consists of three different phonemes, hand posture, location of the hand, and the hand movement. The number of phonemes is limited for the sign language, however, an unlimited number of words can be built from the phonemes. We use the phonemes as the basic units to represent a sign-wrod, and this strategy has the advantage for a further increase of the vocabulary size. We segment the sign-wrod to a sequence of hold and movement segments. The hold segment is analyzed and represented in terms of the phonemes of the hand posture and location. The movement segment is also analyzed and converted to the phoneme of the hand movement. A hand gesture is composed of a sequence of the phonemes. To recognize a dynamic hand gesture, we select the most probable HMM model which represents the specific gesture. In the experiments, we choose twenty Taiwan Sign Language (TSL) sentences for our system to recognize, and collect the sign-language videos made by different signers. The experimental results demonstrate that our system achieves a good performance of sign-word recognition accuracy of 94％ and sentence recognition accuracy of 83.3％.