Abstract
We present an interactive oral rehabilitation system, which detects bites or licks based on the mouth and tongue movements with the aid of Kinect. In order to perform oral physical therapy, our system goads the user into exercising their oral muscles by playing games. The user will get scores when he/she bites or licks the virtual foods falling from the sky. The movements of lips are measured to detect the bite action, and different strategies are designed to locate the tip-of-tongue for various face orientations so that the lick action can be detected more accurately. Our system is designed to provide different levels of therapy according to the rehabilitation situation of each patient in real-time. Also, the experimental results show that our method can be successfully applied to different users.