Abstract
The demand for multimedia applications is increasing even beyond the capabilities of best-effort transmission networks. Therefore, the trend is toward constructing a content-oriented multimedia server that is capable of handling high volumes of content as well as of fulfilling high performance and various user preference requirements. Researchers have been trying to integrate context and content for multimedia mining and management, which is crucial for multimedia communication. The attention analysis of multimedia data is challenging since different models have to be constructed according to different attention characteristics. Effectively measuring the user attention on the videos is an important task in many multimedia applications, including multimedia information retrieval, users-content interaction, and multimedia searching. This thesis analyzes how people are excited about the watched video content and proposes a content-driven attention ranking strategy which enables the client users to iteratively browse the video according to their preference. The proposed attention rank (AR) algorithm, which is extended from the Google PageRank algorithm that sorts the websites based on the importance, can effectively measure the user interest (UI) level for each video frame. The degree of attention is derived by integrating the object-based visual attention model (VAM) with the contextual attention model (CAM), which can reliably take advantage of the human perceptual characteristics, and effectively identify the user-attentive video content. This thesis presents a method to integrate the content and context for sports video understanding. On one hand, visual information is the most intuitive feature of the human perception system. Modeling the visual attention provides a good solution for a better understanding of the video content. The considered visual features includes spatial, temporal, and facial feature maps. On the other hand, the game stat information in the sports video is the most of the subscribers are interested in. The captions embedded in sports video programs represent the key information of the video content. Taking advantage of prior implicit knowledge about sports videos, we proposed an automatic context extraction and interpretation system that can be used to tag the in-game stats for providing the on-going game situation to subscribers. The information of users’ feedback is utilized in re-ranking procedure to further improve the retrieving accuracy. A higher AR represents a stronger user interest. The AR is affected by two factors: intra-AR and inter-AR. In a frame-based analysis, the intra-AR of each frame is based on its visual and contextual attention characteristics. If there are many high-attention objects contained in a frame with a high-interest contextual description, it is highly probable that that frame has a high AR. From an event-based analysis scenario viewpoint, the inter-AR of each frame is affected by the relevant key-frames which are located in the same event.