Efficient Spatiotemporal Representation Learning for Pain Intensity Estimation from Facial Expressions

Experiencing pain is associated with actual or potential tissue damage. Therefore, pain can be considered as an indicator of health condition. Reliable assessment of pain contributes to diagnosing diseases in their early stages to avoid chronic pain syndrome and improving life quality. Pain is a subjective phenomenon that its nature is understood through life experiences. In clinical practice, self-reporting is the gold-standard of pain assessment. Due to the subjective and complex nature of pain, self-reports may not be a reliable assessment approach. Furthermore, it cannot be used for uncommunicative people. On the other hand, observer’s reports of pain are also subjected to errors and biases, and cannot be used for continuous monitoring purposes. Therefore, it is essential to develop automatic pain assessment methodologies to obtain objective information regarding the health condition of the patient. According to medical evidence, facial expressions are a valid indicator of pain. Hence, effective representations of facial expressions can contribute to automatic pain assessment. According to medical evidence, facial expressions are a valid indicator of pain. Hence, effective representations of facial expressions can contribute to automatic pain assessment.

In this thesis, we concentrate on analyzing the facial expressions of pain to estimate pain intensity levels. Due to strong correlations and similarities between facial expressions, direct interpretation of the pain intensity levels is a non-trivial task. Subtle variations of the facial expressions differentiate pain intensity levels from each other. Therefore, we propose deep spatiotemporal representation learning methods to encode different ranges of variations of the face. Specifically, we design novel network architectures and develop learning strategies to effectively capture subtle facial spatiotemporal variations. To address data scarcity in pain assessment from facial expressions, we also present data-efficient machine learning-based models to enhance the performance of automatic pain assessment methods.