Abstract
While the size of multimedia database increases, the demand for efficient browsing and archiving for multimedia data becomes more and more urgent. Video data exhibits a great variety through the co-existence of both audio and visual signals, and therefore, it is necessary to incorporate both audio and visual information into video analysis in order to cope better with human perceptibility. Among them, sport video is one of the major categories, which is globally widespread and draws large audiences. This paper aims to extract baseball game highlights based on audio-motion integrated cues. In order to better describe different audio and motion characteristics in baseball game highlights, we propose a novel representation method based on likelihood model. The proposed likelihood model measures the “likeliness” of low-level audio features and motion features to a predefined audio types and motion patterns, respectively. We will show that the proposed feature representation indeed improves the reliability of using low-level audio/motion features to interpret the highlight. Next, we obtain an integrated feature representation by fusing the audio and motion likelihood models symmetrically. Finally, we employ Hidden Markov Model (HMM) to model and detect the transition of the integrated representation for highlight segments. A series of experiments have been conducted on a 12-hours video database to demonstrate the effectiveness of our proposed method and show that the proposed framework achieves promising results over a variety of baseball game sequences.