Catálogo de publicaciones - libros

The goal of most image based rendering systems can be stated as follows: given a set of pictures taken from various vantage points, synthesize the image that would be obtained from a novel viewpoint. In this paper we present a novel approach to view synthesis which hinges on the observation that human viewers tend to be quite sensitive to the motion of features in the image corresponding to intensity discontinuities or edges. Our system focuses its efforts on recovering the 3D position of these features so that their motions can be synthesized correctly. In the current implementation these feature points are recovered from image sequences by employing the epipolar plane image (EPI) analysis techniques proposed by Bolles, Baker, and Marimont. The output of this procedure resembles the output of an edge extraction system where the edgels are augmented with accurate depth information. This method has the advantage of producing accurate depth estimates for most of the salient features in the scene including those corresponding to occluding contours. We will demonstrate that it is possible to produce compelling novel views based on this information.

The paper will also describe a principled approach to reasoning about the 3D structure of the scene based on the quasi-sparse features returned by the EPI analysis. This analysis allows us to correctly reproduce occlusion and disocclusion effects in the synthetic views without requiring dense correspondences. Importantly, the technique could also be used to analyze and refine the 3-D results returned by range finders, stereo systems or structure from motion algorithms. Results obtained by applying the proposed techniques to actual image data sets are presented.

The lack of eye contact in desktop video teleconferencing substantially reduces the effectiveness of video contents. While expensive and bulky hardware is available on the market to correct eye gaze, researchers have been trying to provide a practical software-based solution to bring video-teleconferencing one step closer to the mass market. This paper presents a novel approach that is based on stereo analysis combined with rich domain knowledge (a personalized face model). This marriage is mutually beneficial. The personalized face model greatly improved the accuracy and robustness of the stereo analysis by substantially reducing the search range; the stereo techniques, using both feature matching and template matching, allow us to extract 3D information of objects other than the face and to determine the head pose in a much more reliable way than if only one camera is used. Thus we enjoy the versatility of stereo techniques without suffering from their vulnerability. By emphasizing a 3D description of the scene on the face part, we synthesize virtual views that maintain eye contact using graphics hardware. Our current system is able to generate an eye-gaze corrected video stream at about 5 frames per second on a commodity PC.

Position disparity between two stereoscopic images, combined with camera calibration information, allow depth recovery. The measurement of position disparity is known to be ambiguous when the scene reflectance displays repetitive patterns. This problem is reduced if one analyzes scale disparity, as in shape from texture, which relies on the deformations of repetitive patterns to recover scene geometry from a single view.

These observations lead us to introduce a new correlation measure based not only on position disparity, but on position and scale disparity. Local scale disparity is expressed as a change in the scale of wavelet coefficients. Our work is related to the spatial frequency disparity analysis of Jones and Malik (ECCV92). We introduce a new wavelet-based correlation measure, and we show its application to stereopsis. We demonstrate its ability to reproduce perceptual results which no other method of our knowledge had accounted for.

In this paper, we formulate the stereo matching problem as a Markov network consisting of three coupled Markov random fields (MRF’s). These three MRF’s model a smooth field for depth/disparity, a line process for depth discontinuity and a binary process for occlusion, respectively. After eliminating the line process and the binary process by introducing two robust functions, we obtain the maximum a posteriori (MAP) estimation in the Markov network by applying a Bayesian belief propagation (BP) algorithm. Furthermore, we extend our basic stereo model to incorporate other visual cues (e.g., image segmentation) that are not modeled in the three MRF’s, and again obtain the MAP solution. Experimental results demonstrate that our method outperforms the state-of-art stereo algorithms for most test cases.

Two central issues in stereo algorithm design are the matching criterion and the underlying smoothness assumptions. In this paper we propose a new stereo algorithm with novel approaches to both issues. We start with a careful analysis of the properties of the continuous (DSI), and derive a new matching cost based on the reconstructed image signals. We then use a symmetric matching process that employs visibility constraints to assign disparities to a large fraction of pixels with minimal smoothness assumptions. While the matching operates on integer disparities, sub-pixel information is maintained throughout the process. Global smoothness assumptions are delayed until a later stage in which disparities are assigned in textureless and occluded areas. We validate our approach with experimental results on stereo images with ground truth.

We investigate a strategy for reconstructing of buildings from multiple (uncalibrated) images. In a similar manner to the Facade approach we first generate a coarse piecewise planar model of the principal scene planes and their delineations, and then use these facets to guide the search for indentations and protrusions such as windows and doors. However, unlike the Facade approach which involves manual selection and alignment of the geometric primitives, the strategy here is fully automatic.

There are several points of novelty: first we demonstrate that the use of quite generic models together with particular scene constraints (the availability of several principal directions) is sufficiently powerful to enable successful reconstruction of the targeted scenes. Second, we develop and refine a technique for piecewise planar model fitting involving sweeping polygonal primitives, and assess the performance of this technique. Third, lines at infinity are constructed from image correspondences and used to sweep planes in the principal directions.

The strategy is illustrated on several image triplets of College buildings. It is demonstrated that convincing texture mapped models are generated which include the main walls and roofs, together with inset windows and also protruding (dormer) roof windows.

In this paper we present a new stereo matching algorithm that produces accurate dense disparity maps and explicitly detects occluded areas. This algorithm extends the original cooperative algorithms in two ways. First, we design a method of adjusting the initial matching score volume to guarantee that correct matches have high matching scores. This method propagates “good” disparity information within or among image segments based on certain disparity confidence measurement criterion, thus improving the robustness of the algorithm. Second, we develop a scheme of choosing local support areas by enforcing the image segmentation information. This scheme sees that the depth discontinuities coincide with the color or intensity boundaries. As a result, the foreground fattening errors are drastically reduced. Extensive experimental results demonstrate the effectiveness of our algorithm, both quantitatively and qualitatively. Comparison between our algorithm and some other representative algorithms is also reported.

Most 3D recording methods generate multiple partial reconstructions that must be integrated to form a complete model. The coarse registration step roughly aligns the parts with each other. Several methods for coarse registration have been developed that are based on matching points between different parts. These methods look for interest points and use a point signature that encodes the local surface geometry to find corresponding points. We developed a technique that is complementary to these methods. Local descriptions can fail or can be highly inefficient when the surfaces contain local symmetries. In stead of discarding these regions, we introduce a method that first uses the Gaussian image to detect planar, cylindrical and conical regions and uses this information to compute the rigid motion between the patches. For combining the information from multiple regions to a single solution, we use a a Hough space that accumulates votes for candidate transformations. Due to their symmetry, they update a subspace of parameter space in stead of a single bin. Experiments on real range data from different views of the same object show that the method can find the rigid motion to put the patches in the same coordinates system.

This paper addresses the problem of large scale multiview registration of range images captured from unknown viewing directions. To reduce the computational burden, we decouple the local problem of pairwise registration on neighboring views from the global problem of distribution of accumulated errors. We define the global problem over the graph of neighboring views, and we show that this graph can be decomposed into a set of cycles such that the optimal transformation parameters for each cycle can be solved in closed form. We then describe an iterative procedure that can be used to integrate the solutions for the set of cycles across the graph. This method for error distribution does not require point correspondences between views, and therefore can be used together with robot odometry or any method of pairwise registration. Experimental results demonstrate the effectiveness of this technique on range images of an indoor facility.

We present a new theoretical method and experimental results for direct recovery of the curvatures, the principal curvature directions, and the surface itself by explicit integration of the Gauss map. The method does not rely on polygonal approximations, smoothing of the data, or model fitting. It is based on the observation that one can recover the surface restoring force from the Gauss map, and (i) applies to orientable surfaces of arbitrary topology (not necessarily closed); (ii) uses only first order linear differential equations; (iii) avoids the use of unstable computations; (iv) provides tools for filtering noise from the sampled data. The method can be used for stable extraction of surfaces and surface shape invariants, in particular, in applications requiring accurate quantitative measurements.