Catálogo de publicaciones - libros

General multi view reconstruction from affine or projective cameras has so far been solved most efficiently using methods of factorizing image data matrices into camera and scene parameters. This can be done directly for affine cameras[] and after computing epipolar geometry for projective cameras []. A notorious problem has been the fact that these factorization methods require all points to be visible in all views. This paper presents alternative algorithms for general affine and projective views of multiple points where a) points and camera centers are computed as the nullspace of one linear system constructed from all the image data b) only three points have to be visible in all views. The latter requirement increases the flexibility and usefulness of 3D reconstruction from multiple views. In the case of projective views and unknown epipolar geometry, an additional algorithm is presented which initially assumes affine views and compensates iteratively for the perspective effects. In this paper affine cameras are represented in a projective framework which is novel and leads to a unified treatment of parallel and perspective projection in a single framework. The experiments cover a wide range of different camera motions and compare the presented algorithms to factorization methods, including approaches which handle missing data.

In this work, we introduce a model-based approach to extracting the silhouette of people in motion from stereo video sequences. To this end, we extend a purely stereo-based approach to tracking people proposed in earlier work. This approach is based on an implicit surface model of the body. It lets us accurately predict the silhouette’s location and, therefore, detect them more robustly. In turn these silhouettes allow us to fit the model more precisely. This allows effective motion recovery, even when people are filmed against a cluttered unknown background. This is in contrast to many recent approaches that require silhouette contours to be readily obtainable using relatively simple methods, such as background subtraction, that typically require either engineering the scene or making strong assumptions.

We demonstrate our approach’s effectiveness using complex and fully three-dimensional motion sequences where the ability to combine stereo and silhouette information is key to obtaining good results.

I address the problem of optimizing projective motion over a minimal set of parameters. Most of the existing works overparameterize the problem. While this can simplify the estimation process and may ensure well-conditioning of the parameters, this also increases the computational cost since more unknowns than necessary are involved.

I propose a method whose key feature is that the number of parameters employed is minimal. The method requires singular value decomposition and minor algebraic manipulations and is therefore straightforward to implement. It can be plugged into most of the optimization algorithms such as Levenberg-Marquardt as well as the corresponding sparse versions. The method relies on the orthonormal camera motion representation that I introduce here. This representation can be locally updated using minimal parameters.

I give a detailled description for the implementation of the two-view case within a bundle adjustment framework, which corresponds to the maximum likelihood estimation of the fundamental matrix and scene structure. Extending the algorithm to the multiple-view case is straightforward. Experimental results using simulated and real data show that algorithms based on minimal parameters perform better than the others in terms of the computational cost, i.e. their convergence is faster, while achieving comparable results in terms of convergence to a local optimum. An implementation of the method will be made available.

This paper proposes a method for recovery of projective shape and motion from multiple images by factorization of a matrix containing the images of all scene points. Compared to previous methods, this method can handle perspective views and occlusions jointly. The projective depths of image points are estimated by the method of Sturm & Triggs [] using epipolar geometry. Occlusions are solved by the extension of the method by Jacobs [] for filling of missing data. This extension can exploit the geometry of perspective camera so that both points with known and unknown projective depths are used. Many ways of combining the two methods exist, and therefore several of them have been examined and the one with the best results is presented. The new method gives accurate results in practical situations, as demonstrated here with a series of experiments on laboratory and outdoor image sets. It becomes clear that the method is particularly suited for wide base-line multiple view stereo.

We present a linear method for self-calibration of a moving rig when no correspondences are available between the cameras. Such a scenario occurs, for example, when the cameras have different viewing angles, different zoom factors or different spectral ranges. It is assumed that during the motion of the rig, the relative viewing angle between the cameras remains fixed and is known. Except for the fixed relative viewing angle, any of the internal parameters and any of the other external parameters of the cameras may vary freely. The calibration is done by linearly computing multilinear invariants, expressing the relations between the optical axes of the cameras during the motion. A solution is then extracted from these invariants. Given the affine calibration, the metric calibration is known to be achieved linearly (e.g. by assuming zero skew). Thus an automatic solution is presented for self calibration of a class of moving rigs with varying internal parameters. This solution is achieved without using any correspondences between the cameras, and requires only solving linear equations.

We study the multi-frame structure from motion problem when the camera translates on a plane with small baselines and arbitrary rotations. This case shows up in many practical applications, for example, in ground robot navigation. We consider the framework for small baselines presented in [], in which a factorization method is used to compute the structure and motion parameters accurately, efficiently and with guaranteed convergence. When the camera translates on a plane, the algorithm in [] cannot be applied because the estimation matrix drops rank, causing the equations to be no longer linear. In this paper, we show how to linearly solve those equations, while preserving the accuracy, speed and convergence properties of the non-planar algorithm. We evaluate the proposed algorithms on synthetic and real image sequences, and compare our results with those of the optimal algorithm. The proposed algorithms are very fast and accurate, have less than 0.3% outliers and work well for small-to-medium baselines and non-planar as well as planar motions.

Given the projection of a sufficient number of points it is possible to algebraically eliminate the camera parameters and obtain view-invariant functions of image coordinates and space coordinates. These single view invariants have been introduced in the past, however, they are not as well understood as their dual multi-view tensors. In this paper we revisit the dual tensors (bilinear, trilinear and quadlinear), both the general and the reference-plane reduced version, and describe the complete set of synthetic constraints, properties of the tensor slices, reprojection equations, non-linear constraints and reconstruction formulas. We then apply some of the new results, such as the dual reprojection equations, for multi-view point tracking under occlusions.

Estimating geometric structure from uncalibrated images accurately enough for high quality rendering is difficult. We present a method where only coarse geometric structure is tracked and estimated from a moving camera. Instead a precise model of the intensity image variation is obtained by overlaying a dynamic, time varying texture on the structure. This captures small scale variations (e.g. non-planarity of the rendered surfaces, small camera geometry distortions and tracking errors). The dynamic texture is estimated and coded much like in movie compression, but parameterized in 6D pose instead of time, hence allowing the interpolation and extrapolation of new poses in the rendering and animation phase. We show experiments tracking and re-animating natural scenes as well as evaluating the geometric and image intensity accuracy on constructed special test scenes.

A common practice when carrying out self-calibration and Euclidean reconstruction from one or more views is to start with a guess at the principal point of the camera. The general belief is that inaccuracies in the estimation of the principal point do not have a significant effect on the other calibration parameters, or on reconstruction accuracy. It is the purpose of this paper to refute that belief. Indeed, it is demonstrated that the determination of the focal length of the camera is tied up very closely with the estimate of the principal point. Small changes in the estimated (sometimes merely guessed) principal point can cause very large changes in the estimated focal length, and the accuracy of reconstruction. In fact, the relative uncertainty in the focal length is inversely proportional to the distance of the principal point to the epipolar line. This analysis is geometric and exact, rather than experimental.

The problem of recovering scene structure and camera motion from images has a number of inherent ambiguities. In this paper, configurations of points and cameras are analyzed for which the image points alone are insufficient to recover the scene geometry uniquely. Such configurations are said to be critical. For two views, it is well-known that a configuration is critical only if the two camera centres and all points lie on a ruled quadric. However, this is only a necessary condition. We give a complete characterization of the critical surfaces for two calibrated cameras and any number of points. Both algebraic and geometric characterizations of such surfaces are given. The existence of critical sets for -view projective reconstruction has recently been reported in the literature. We show that there are critical sets for -view Euclidean reconstruction as well. For example, it is shown that for any placement of three calibrated cameras, there always exists a critical set consisting of any number of points on a fourth-degree curve.