Abstract
In this thesis, we propose a novel system to estimate the depth of outdoor scenes from a video sequence. According to the characteristics of a video, our approach considers more information in the temporal domain than the traditional depth reconstruction methods. We perform Structure From Motion (SfM) on images sampled from a video by extracting and matching a set of Scale Invariant Feature Transform (SIFT) feature points. This provides some camera information, including 3D translation and rotation, for all the images. Then, we compute the constrained optical flow between selected scenes so that we can solve an over-constrained linear system to estimate the depth map for each frame. After that, mean shift image segmentation [11] is applied to alleviate the estimation problem with textureless regions and outlier points. The initial depth maps can be done by incorporating predicted depth maps, segmentation results, and some geometric constraints. This initial depth map becomes the data term of our pixel-based and region-based Markov Random Field formulation for depth map estimation. By minimizing the associated MRF energy function for each frame, we can refine the depth maps to achieve visually pleasing, detail-preserving and temporally consistent depth estimation results.