Abstract
This dissertation studies quality enhancement and assessment for image/video resizing. To achieve high-quality reconstruction of high-resolution (HR) details for a low-resolution (LR) image/video, super-resolution (SR) has proven to be an efficient approach. Particularly, learning-based SR schemes usually show superior performance, compared to conventional multi-frame SR approach. In part-I, we address three issues in learning-based image and video SR. The first task for real-world SR applications is to achieve simultaneous SR and deblocking for a highly compressed image. In our method, we propose to learn image sparse representations for modeling the relationship between low and high-resolution image patches in terms of the learned dictionaries for image patches with and without blocking artifacts, respectively. As a result, image SR and deblocking can be simultaneously achieved via sparse representation and MCA (morphological component analysis)-based dictionary classification. In this way, the learned dictionary can be successfully classified into two sub-dictionaries with and without blocking artifacts. Second, we propose a two-step face hallucination. Since the coefficients for representing a LR face image with LR dictionary is unreliable due to insufficient observed information, we propose a maximum-a-posterior (MAP) estimator to re-estimate the coefficients, which significantly improves the visual quality of the reconstructed face. Besides, the facial parts (i.e., eyes, nose and mouth) are further refined using the proposed basis selection method for overcomplete nonnegative matrix factorization (ONMF) dictionary to eliminate unnecessary information in basis. Third, we propose a texture-synthesis-based video SR method, in which a novel dynamic texture synthesis (DTS) scheme is proposed to render the reconstructed HR details in a temporally coherent way, which effectively addresses the temporal incoherence problem caused by traditional texture synthesis based image SR methods. To reduce the computational complexity, our method only performs the texture synthesis-based SR on a selected set of key-frames, while the HR details of the remaining non-key-frames are simply predicted using the bi-directional overlapped block motion compensation. After all frames are upscaled, the proposed DTS-SR is applied to maintain the temporal coherence in the HR video. The second part of this dissertation is quality assessment for image/video resizing techniques. Image/video retargeting algorithm has been comprehensively studied in past decade. However, there is no accurate objective quality assessment algorithm for image/video retargeting. We therefore propose a novel full-reference objective metric for automatically assessing visual quality of a retargeted image based on perceptual geometric distortion and information loss. The proposed metric measures the geometric distortion of retargeted images based on the local variance of SIFT flow vector fields. A visual saliency map is further derived to characterize human perception of the geometric distortion. Besides, the information loss in a retargeted image, which is estimated based on the saliency map, is also taken into account in the proposed metric. Furthermore, we extend the SIFT flow estimation to temporal domain for video retargeting quality assessment, in which the local temporal distortion can be measured by analyzing the local variance of the SIFT map vector fields. Experimental results demonstrate the proposed metrics for image and video retargeting significantly outperform existed state-of-the-art metrics.