Catálogo de publicaciones - libros

This paper describes an interactive system for creating geometric models from many uncalibrated images of architectural scenes. In this context, we must solve the structure from motion problem given only few and noisy feature correspondences in non-sequential views. By exploiting the strong constraints obtained by modelling a map as a single affine view of the scene, we are able to compute all 3D points and camera positions simultaneously as the solution of a set of linear equations. Reconstruction is achieved without making restrictive assumptions about the scene (such as that reference points or planes are visible in all views). We have implemented a practical interactive system, which has been used to make large-scale models of a variety of architectural scenes. We present quantitative and qualitative results obtained by this system.

Depth extraction with a mobile stereo system is described. The stereo setup is precalibrated, but the system extracts its own motion. Emphasis lies on the integration of the motion and stereo cues. It is guided by the relative confidence that the system has in these cues. This weighing is fine-grained in that it is determined for every pixel at every iteration. Reliable information spreads fast at the expense of less reliable data, both in terms of spatial communication and in terms of exchange between cues. The resulting system can handle large displacements, depth discontinuities and occlusions. Experimental results corroborate the viability of the approach.

Lens distortions in off-the-shelf or wide-angle cameras block the road to high accuracy Structure and Motion Recovery (SMR) from video sequences. Neglecting lens distortions introduces a systematic error buildup which causes recovered structure and motion to bend and inhibits turntable or other loop sequences to close perfectly. Locking back onto previously reconstructed structure can become impossible due to the large drift caused by the error buildup. Bundle adjustments are widely used to perform an ultimate post-minimization of the total reprojection error. However, the initial recovered structure and motion needs to be close to optimal to avoid local minima. We found that bundle adjustments cannot remedy the error buildup caused by ignoring lens distortions. The classical approach to distortion removal involves a preliminary distortion estimation using a calibration pattern, known geometric properties of perspective projections or only 2D feature correspondences. Often the distortion is assumed constant during camera usage and removed from the images before applying SMR algorithms. However, lens distortions can change by zooming, focusing and temperature variations. Moreover, when only the video sequence is available preliminary calibration is often not an option. This paper addresses all fore-mentioned problems by sequentially recovering lens distortions together with structure and motion from video sequences without tedious pre-calibrations and allowing lens distortions to change over time. The devised algorithms are fairly simple as they only use linear least squares techniques. The unprocessed video sequence forms the only input and no severe restrictions are placed on viewed scene geometry. Therefore, the accurate recovery of structure and motion is fully automated and widely applicable. The experiments demonstrate the necessity of modeling lens distortions to achieve high accuracy in recovered structure and motion.

In this paper, the geometry of a general class of projections from ℝ to ℝ ( < ) is examined, as a generalization of classic multiple view geometry in computer vision. It is shown that geometric constraints that govern multiple images of hyperplanes in ℝ, as well as any incidence conditions among these hyperplanes (such as inclusion, intersection, and restriction), can be systematically captured through certain rank conditions on the so-called multiple view matrix. All constraints known or unknown in computer vision for the projection from ℝ to ℝ are simply instances of this result. It certainly simplifies current efforts to extending classic multiple view geometry to dynamical scenes. It also reveals that since most new constraints in spaces of higher dimension are , the rank conditions are a natural replacement for the traditional multilinear analysis. We also demonstrate that the rank conditions encode extremely rich information about dynamical scenes and they give rise to fundamental criteria for purposes such as stereopsis in -dimensional space, segmentation of dynamical features, detection of spatial and temporal formations, and rejection of occluding T-junctions.

For 3-D video applications, dense depth maps are required. We present a segment-based structure-from-motion technique. After image segmentation, we estimate the motion of each segment. With knowledge of the camera motion, this can be translated into depth. The optimal depth is found by minimizing a suitable error norm, which can handle occlusions as well. This method combines the advantages of motion estimation on the one hand, and structure-from-motion algorithms on the other hand. The resulting depth maps are pixel-accurate due to the segmentation, and have a high accuracy: depth differences corresponding to motion differences of 1/8 of a pixel can be recovered.

Establishing point correspondences between images is a key step for 3D-shape computation. Nevertheless, shape extraction and point correspondence are treated, usually, as two different computational processes. We propose a new method for solving the correspondence problem between points of a fully uncalibrated scaled-orthographic image sequence. Among all possible point selections and permutations, our method chooses the one that minimizes the fourth singular value of the observation matrix in the factorization method. This way, correspondences are set such that shape and motion computation are optimal. Furthermore, we show this is an optimal criterion under bounded noise conditions.

Also, our formulation takes feature selection and outlier rejection into account, in a compact and integrated way. The resulting combinatorial problem is cast as a concave minimization problem that can be efficiently solved. Experiments show the practical validity of the assumptions and the overall performance of the method.

The camera registration extracted from feature based stereo is usually considered sufficient to accurately localize the 3D points. However, for natural scenes the feature localization is not as precise as in man-made environments. This results in small camera registration errors. We show that even very small registration errors result in large errors in dense surface reconstruction.

We describe a method for registering images to the inaccurate surface model. This gives small, but crucially important improvements to the camera parameters. The new registration gives dramatically better dense surface reconstruction.

This paper presents a novel approach to camera motion parametrization for the structure and motion problem. In a model-based framework, the hypothesis of (relatively) continuous and smooth sensor motion enables to reformulate the motion recovery problem as a global curve estimation problem on the camera path. Curves of incremental complexity are fitted using model selection to take into account incoming image data. No first estimate guess is needed. The use of modeling curves lead to a meaningful description of the camera trajectories, with a drastic reduction in the number of degrees of freedom. In order to characterize the behaviour and performances of the approach, experiments with various long video sequences, both synthetic and real, are undertaken. Several candidate curve models for motion estimation are presented and compared, and the results validate the work in terms of reconstruction accuracy, noise robustness and model compacity.

In this paper, a Bayesian self-calibration approach is proposed using sequential importance sampling (SIS). Given a set of feature correspondences tracked through an image sequence, the joint posterior distributions of both camera extrinsic and intrinsic parameters as well as the scene structure are approximated by a set of samples and their corresponding weights. The critical motion sequences are explicitly considered in the design of the algorithm. The probability of the existence of the critical motion sequence is inferred from the sample and weight set obtained from the SIS procedure. No initial guess for the calibration parameters is required. The proposed approach has been extensively tested on both synthetic and real image sequences and satisfactory performance has been observed.

Metric reconstruction of a scene viewed by an uncalibrated camera undergoing an unknown motion is a fundamental task in computer vision. To obtain accurate results all the methods rely on bundle adjustment, a nonlinear optimization technique which minimizes the reprojection error over the structural and camera parameters. Bundle adjustment is optimal for normally distributed measurement noise, however, its performance depends on the starting point. The initial solution is usually obtained by solving a linearized constraint through a total least squares procedure, which yields a biased estimate. We present a more balanced approach where in main computational modules of an uncalibrated reconstruction system, the initial solution is obtained from a statistically justified estimator which assures its unbiasedness. Since the quality of the new initial solution is already comparable with that of the result of bundle adjustment, the burden on the latter is drastically reduced while its reliability is significantly increased. The performance of our system was assessed for both synthetic data and standard image sequences.