Catálogo de publicaciones - libros

In this paper we present a vision system that analyzes the gestures of a noted conductor conducting a real orchestra, a different approach from previous work that allowed users to conduct virtual orchestras with prerecorded scores. We use a low-resolution video sequence of a live performance of the Boston Symphony Orchestra, and we track the conductor’s right hand. The tracker output is lined up with the output of an audio beat tracker run on the same sequence. The resulting analysis has numerous implications for the understanding of musical expression and gesture.

This paper proposes a comparison of two motion estimation algorithms for crowd scene analysis in video sequences. The first method uses the local gradient supported by neighbouring topology constraints. The second method makes use of descriptors extracted from points lying at the maximum curvature along Canny edges. Performance is evaluated using real-world video sequences, providing the reader with a quantitative comparison of the two methods.

This paper presents a real-time emotion detection system capable of identifying seven affective states: agreeing, concentrating, disagreeing, interested, thinking, unsure, and angry from a near infrared video stream. A Viola Jones face detector is trained to locate the face within the frame. The Active Appearance Model is then used to place 23 landmark points around key areas of the eyes, brows, and mouth. A prioritized binary decision tree then detects, based on the actions of these key points, if on of the seven emotional states occurs as frames pass. The completed system runs accurately and seamlessly on an Intel Pentium IV, 2.8 GHz processor with 512 MB of memory, achieving a real-time frame rate of 36 frames per second.

Video event understanding requires a formalism that can model complex logical temporal and spatial relations between composing sub-events. In this paper we argue that the Petri-Net is such a formalism. We go on to define a methodology for constructing Petri-Net event models from semantic descriptions of events in two well known video event ontology standards, VERL and CASE.

In this paper, we present a facial expression recognition method using feature-adaptive motion energy analysis. Our method is simplicity-oriented and avoids complicated face model representations or computationally expensive algorithms to estimate facial motions. Instead, the proposed method uses a simplified action-based face model to reduce the computational complexity of the entire facial expression analysis and recognition process. Feature-adaptive motion energy analysis estimates facial motions in a cost-effective manner by assigning more computational complexity on selected discriminative facial features. Facial motion intensity and orientation evaluation are then performed accordingly. Both facial motion intensity and orientation evaluation are based on simple calculations by exploiting a few motion energy values in the difference image, or optimizing the characteristics of feature-adaptive facial feature regions. For facial expression classification, a computationally inexpensive decision tree is used since the information gain heuristics of ID3 decision tree forces the classification to be done with minimal Boolean comparisons. The feasibility of the proposed method is shown through the experimental results as the proposed method recognized every facial expression in the JAFFE database by up to 75% with very low computational complexity.

We present a novel boosting algorithm where temporal consistency is addressed in a short-term way. Although temporal correlation of observed data may be an important cue for classification (e.g. of human activities) it is seldom used in boosting techniques. The recently proposed Temporal AdaBoost addresses the same problem but in a heuristic manner, first optimizing the weak learners without temporal integration. The classifier responses for past frames are then averaged together, as long as the total classification error decreases.

We extend the GentleBoost algorithm by modeling time in an explicit form, as a new parameter during the weak learner training and in each optimization round. The time consistency model induces a fuzzy decision function, dependent on the temporal support of a feature or data point, with added robustness to noise. Our temporal boost algorithm is further extended to cope with multi class problems, following the JointBoost approach introduced by Torralba We can thus (i) learn the parameters for all classes at once, and (ii) share features among classes and groups of classes, both in a temporal and fully consistent manner.

Finally, the superiority of our proposed framework is demonstrated comparing it to state of the art, temporal and non-temporal boosting algorithms. Tests are performed both on synthetic and 2 real challenging datasets used to recognize a total of 12 different human activities.

In comparison with 2D face images, 3D face models have the advantage of being illumination and pose invariant, which provides improved capability of handling changing environments in practical surveillance. Feature detection, as the initial process of reconstructing 3D face models from 2D uncalibrated image sequences, plays an important role and directly affects the accuracy and robustness of the resulting reconstruction. In this paper, we propose an automated scene-specific selection algorithm that adaptively chooses an optimal feature detector according to the input image sequence for the purpose of 3D face reconstruction. We compare the performance of various feature detectors in terms of accuracy and robustness of the sparse and dense reconstructions. Our experimental results demonstrate the effectiveness of the proposed selection method from the observation that the chosen feature detector produces 3D reconstructed face models with superior accuracy and robustness to image noise.

A novel method, called dynamic component deforming model, is proposed to reconstruct the face shape from a 2D image based on feature points. Assuming that human face belongs to a linear class, principal components learned from a 3D face database are used in order to constrain the results. Different from the fixed components used in the traditional methods, the significance of each component is investigated while the most correlative ones are selected as the basic space. This novel representation is able to fit a more exact 3D shape for an individual than the known methods as the useless data are excluded. Comparison results show that the proposed method achieves good results on both contrived data with known ground truth together with real photographs.

An efficient and robust method to locate eyes and lip corners under various illumination conditions is presented in this paper. First, a global illumination balance method is presented to compensate for variations in illumination and accentuate facial feature details. Next, a human face is distinguished from background based on Haar-like features. After that, eyes are found based on their intensity characteristics and Haar-like features. The lip region is estimated using the positions and sizes of face and eyes. Then, a novel local illumination balance technique, based on an integral projection analysis, is proposed to correct for the non-uniform illumination in the lip region. Finally, lip corners are detected based on their intensity and geometric characteristics. Encouraging results have been obtained using the proposed method.

Despite significant amount of research on automatic classification of facial expressions, recognizing a facial expression remains a complex task to be achieved by a computer vision system. Our approach is based on a close look at the mechanisms of the human visual system, the best automatic facial expression recognition system yet. The proposed model is made for the classification of the six basic facial expressions plus on static frames based on the permanent facial features deformations using the Transferable Belief Model. The aim of the proposed work is to understand how the model behaves in the same experimental conditions as the human observer, to compare their results and to identify the missing informations so as to enhance the model performances. To do this we have given our TBM based model the ability to deal with partially occluded stimuli and have compared the behavior of this model with that of humans in a recent experiment, in which human participants had to classify the studied expressions that were randomly sampled using Gaussian apertures. Simulations show first the suitability of the TBM to deal with partially occluded facial parts and its ability to optimize the available information to take the best possible decision. Second they show the similarities of the human and model observers performances. Finally, we reveal important differences between the use of facial information in the human and model observers, which open promising perspectives for future developments of automatic systems.