Computational Modeling for Visual Attention Analysis

Visual scenes typically contain massive amounts of content that cannot be processed in a short time due to the limited processing capacity of the human visual system. The term, visual attention, is a biologically inspired and psychologically driven mechanism that works by selecting visually relevant information and filtering out the redundant contents. To duplicate the attention mechanism to facilitate compute vision tasks, computational modeling for visual attention analysis is crucial to suggest the salient regions on visual scenes by the human visual system.

This thesis is a thorough summary of the main subjects around computational modeling for visual attention analysis, consisting of several published papers corresponding to my research progress. First, the data preparation for computational modeling will be introduced, including eye movement data, eye tracking data collection and eye tracking datasets facilitating the evaluation of computational modeling of visual attention. Second, computational models for visual attention analysis, or saliency models, are presented from traditional unsupervised methods to deep saliency models. Third, the subject about saliency integration will be illustrated that unifies multiple saliency maps from the multiple candidate saliency models for better accuracy.

The contributions of this study are three folds. Firstly, we collect a task-driven eye tracking dataset for visual attention analysis. Secondly, we propose three saliency models for in-depth investigation in modeling visual attention, including an unsupervised model using the bi-directional propagation method, a Convolutional Neural Networks based model by connecting the Dense Conditional Random Fields for multi-scale saliency refinement, and a Convolutional Neural Networks based model with cascade Conditional Random Fields for joint model training. Thirdly, we propose a saliency integration method and conduct comprehensive experiments and analysis on the topic. Finally, we summarize the contributions of the work and propose the potential applications of saliency models and the extended saliency related topics to boost applications of saliency approaches on other computer vision topics.