Abstract
Nowadays, emergency department are often considered as the most efficient ways to seek medical care. However, to allocate the healthcare resource effectively, triage classification system plays an important role in assessing the severity of illness of the boarding patient at emergency department. There are some factors listed in Taiwan triage and acuity scale (TTAS) about triage classification system. And the self-report pain intensity numerical-rating scale (NRS) is one of the major modifiers of the current triage system based on the TTAS. In clinical practice, physicians and nurses have noticed the difficulty in the systematic implementation of this instrument especially for elderly people, foreigners, or patients with a low education level. This often leads to the triage nurses would select the level through his/her own observations instead of soliciting an answer from the patient. These ways would create a deviation on the consistency and validity of the triage classification system. In this paper, we have cooperation with emergency physicians in Linkou Chang Gung Memorial Hospital. We extract the multimodal behavioral signal of facial expression and vocal characteristics from patients, and model these behaviors by using machine learning models of CNN and LSTM respectively. The experimental results show that the accuracy of 77.1% and 55.7%, respectively, in the two and three classes of pain recognition. Further, in the experimental analysis, we also found that it had significant relationship with facial expression and vocal characteristics of patients.