Abstract
In this thesis, we introduce a vision-based continuous sign language recognition to recognize sign language sentences in a simple background. The system consists of four modules: feature extraction, product hidden Markov model (PHMM) training, sign words recognition using the PHMMs. To allow real-time moving hand tracking and hand shape extraction, the signer wears gloves with different colors. CamShift algorithm is used to track the moving hands gesture in a video. We apply the 7Hu moment and orientation of major axis to characterize the hand shape. After havingextracted the feature, we train our system using PHMM. Then, we use the hand location to roughly segment the continuous sign. After rough segmentation, we apply the hand-shape-based segmentation to divide the CSR image sequence into image sub-sequences, and then use the trained PHMM to recognize the isolated sign word.In the experiments, we choose 40 Taiwan Sign Language (TSL) sign words, and collect the sign language videos made by different signers. The experimental results demonstrate that our system achieves a good performance of sign-word recognition accuracy of 94.04%. In another experiments, we collect 3 TSL sentences which consist of 18~23 sign-words. The experimental results show that the average signspotting recall rate is 74.5% and precision rate is 89%.