Vision-based 3D perception for mixed reality applications

Mixed reality (MR) refers to merging of real and virtual worlds using computer graphics. MR is implemented by overlaying virtual objects or scenes to the user’s field of view with optical or video see-through display systems, including wearable headsets and smartphones. A major challenge in MR is the alignment of the virtual objects with the visual information perceived from the real world. Accurate alignment is necessary to create realistic visualizations and immersive user experience, and it can only be achieved by exploiting the 3D structure of the environment. Another challenge is automatic analysis of the 3D scene that cannot be fully resolved by using 2D image recognition techniques. In this project, we focus on vision-based 3D perception that provides the basis for accurate alignment as well as for recognition. The aim is to develop novel methods for 3D sensing, modeling, and rendering, and build demonstrators to validate the performance of these methods in MR applications.

Strategic research project of the University of Oulu

Focus institute: Infotech Oulu

Faculty: Faculty of Information Technology and Electrical Engineering (ITEE)