Gaze on lips?

During communication, speechreading (visual speech recognition) and facial expressions are inherently and unconsciously utilized to support auditory speech recognition. Speechreading is necessary especially in adverse listening conditions and for compensating hearing difficulties. For the best speechreading performance, there is, so far, only scarce international research evidence on where one should look at and for how long. However, some previous studies have indicated that speechreading skills can be trained but there is a need to explore the issue more closely.

In Gaze on lips? we shall construct a speechreading test and an application for practising speechreading. Using them, we research how children use their gaze when they speechread silently produced words and sentences and how speechreading skills can be improved. Our results will help to make training of speechreading more effective e.g., in speech and language therapy. A speechreading application The Optic Track has been constructed, and data are being collected with the research version of it.

In addition, the effectiveness of computer vision algorithms on speechreading will be investigated. Separate data are collected in CMVS for that purpose. The aim is to develop further computer vision methods constructed for visual speech recognition.