Perspective Analysis of Photographs and Its Applications

Abstract

Perspective of a photo provides visual cue of depth perception of the 3D space.It is independent of monocular or binocular vision

one can utilize those cues to recognize and reconstruct 3D structure from a single image.The human perception of perspective is known to be heavily depending on learning and relies on priors a lot. Such priors not only affects the 3D recognition process of the human, but the evaluation of subjective quality of images.Understanding and utilizing such priors is, therefore, a key of building good perspective analysis and manipulation algorithms and applications.Such applications include camera calibration and pose estimation, 3D reconstruction from single or multiple images, 3D-aware object and scene retrieval, editing, composition, and a lot more.For the perspective analysis of photographs, we propose a single image based camera calibration method.From an input image we estimate intrinsic parameters of the camera and the relative orientation of the camera with respect to the scene.A set of priors, both on the 3D scene with man-made structures and the camera itself, are first defined.Then an energy minimization framework is build upon those priors to estimate intrinsic parameters and the relative orientation of the camera.With the proposed priors our camera calibration method achieves state-of-the-art performance.Based on our camera calibration method, we present two applications

upright adjustment and perspective-aware image retrieval and composition.Upright adjustment is an automatic perspective adjustment method, which aims at straightening up slanted man-made structures in an input image to improve its perceptual quality.We propose a set of criteria for upright adjustment based on human perception studies.With a given input photo, the criteria is evaluated based on 3D structure of the scene estimated via our camera calibration method.Then an optimization framework is developed which yields an optimal adjustment making the photo perceptually pleasing.Our method is fully automatic, works reliably on a wide range of images, and is comprehensively evaluated through qualitative user-studies.We also present a perspective-aware image retrieval and composition system.For the image retrieval, we propose a simple image feature that effectively encapsulates estimated 3D structures of images.Based on the proposed image feature, we build an image dataset with various types of objects.In the run-time, our system is able to retrieve object images with similar 3D structures from an given image query.Finally we present an image composition method that efficiently creates composites with well-aligned 3D structures between the target image and the object image, making perceptually plausible composition results.