Catálogo de publicaciones - libros

This paper presents a set of techniques that allow generating a class of testbeds that can be used to test recurrent neural networks’ capabilities of integrating information in time. In particular, the testbeds allow evaluating the capability of such models, and possibly other architectures and algorithms, of (a) categorizing different time series, (b) anticipating future signal levels on the basis of past ones, and (c) functioning robustly with respect to noise and other systematic random variations of the temporal and spatial properties of the input time series. The paper also presents a number of analysis tools that can be used to understand the functioning and organization of the dynamical internal representations that recurrent neural networks develop to acquire the aforementioned capabilities, including periodicity, repetitions, spikes, and levels and rates of change of input signals. The utility of the proposed testbeds is illustrated by testing and studying the capacity of Elman neural networks to predict and categorize different signals in two exemplary tasks.

We introduce information-theoretic tools that can be used in an autonomous agent for constructing an internal predictive model based on event anticipation. This model relies on two different kinds of predictive relationships: time-delay relationships, where two events are related by a nearly constant time-delay between their occurrences; and contingency relationships, where proximity in time is the main property. We propose an anticipation architecture based on these tools that allows the construction of a relevant internal model of the environment through experience. Its design takes into account the problem of handling different time scales. We illustrate the effectiveness of the tools proposed with preliminary results about their ability to identify relevant relationships in different conditions. We describe how these principles can be embedded in a more complex architecture that allows action-decision making according to reward expectation, and handling of more complex relationships. We conclude by discussing issues that were not addressed yet and some axis for future investigations.

This chapter outlines an approach to building robots with anticipatory behavior based on analogies with past episodes. Anticipatory mechanisms are used to make predictions about the environment and to control selective attention and top-down perception. An integrated architecture is presented that perceives the environment, reasons about it, makes predictions and acts physically in this environment. The architecture is implemented in an AIBO robot. It successfully finds an object in a house-like environment. The AMBR model of analogy-making is used as a basis, but it is extended with new mechanisms for anticipation related to analogical transfer, for top down perception and selective attention. The bottom up visual processing is performed by the IKAROS system for brain modeling. The chapter describes the first experiments performed with the AIBO robot and demonstrates the usefulness of the analogy-based anticipation approach.

Regardless of complex, unknown, and dynamically-changing environments, living creatures can recognize situated environments and behave adaptively in real-time. However, it is impossible to prepare optimal motion trajectories with respect to every possible situations in advance. The key concept for realizing the environment cognition and motor adaptation is a context-based elicitation of constraints which are canalizing well-suited sensorimotor coordination. For this aim, in this study, we propose a polymorphic neural networks model called CTRNN+NM (CTRNN with neuromodulatory bias). The proposed model is applied to two dimensional arm-reaching movement control under various viscous force fields. The parameters of the networks are optimized using genetic algorithms. Simulation results indicate that the proposed model inherits high robustness even though it is situated in unexperienced environments.

This paper presents the first basic principles, implementation and experimental results of what could be regarded as a new approach to reinforcement learning, where agents—physical robots interacting with objects and other agents in the real world—can learn to anticipate rewards using their sensory inputs. Our approach does not need discretization, notion of events, or classification, and instead of learning rewards for the different possible actions of an agent in all the situations, we propose to make agents learn only the main situations worth avoiding and reaching. However, the main focus of our work is not reinforcement learning as such, but modeling cognitive development on a small autonomous robot interacting with an “adult” caretaker, typically a human, in the real world; the control architecture follows a Perception-Action approach incorporating a basic homeostatic principle. This interaction occurs in very close proximity, uses very coarse and limited sensory-motor capabilities, and affects the “well-being” and affective state of the robot. The type of anticipatory behavior we are concerned with in this context relates to both sensory and reward anticipation. We have applied and tested our model on a real robot.

The role of expectation in listening and composing music has drawn much attention in music cognition since about half a century ago. In this paper, we provide a first attempt to model some aspects of musical expectation specifically pertained to short-time and working memories, in an anticipatory framework. In our proposition anticipation is the mental realization of possible predicted actions and their effect on the perception of the world at an instant in time. We demonstrate the model in applications to automatic improvisation and style imitation. The proposed model, based on cognitive foundations of musical expectation, is an active model using reinforcement learning techniques with multiple agents that learn competitively and in collaboration. We show that compared to similar models, this anticipatory framework needs little training data and demonstrates complex musical behavior such as long-term planning and formal shapes as a result of the anticipatory architecture. We provide sample results and discuss further research.

Competitive distributed systems pose a challenge to trust modeling due to the dynamic nature of these systems (e.g. electronic auctions) and the unreliability of self-interested agents. We propose a trust model which does not assume a concrete cognitive model for other agents that an agent may interact with, but uses the discrepancy between the information provided by other agents and its own experience in order to anticipate their actions. By anticipating the behavior of other agents, an agent is able to adapt more effectively to changes in the environment for its own benefit.

In this paper we present a conceptualization and a formalization to define agents’ behaviors (as exhibited in agent to agent interactions), via an extension of Petri Nets, and show how behaviors of different agents can be aligned. We explain why these agents can be considered anticipatory, and the link between Business Information Systems and anticipatory systems is elaborated. We show that alignment is a state anticipatory mechanism, where predictions about future states directly influence current behavioral decision making. This results in faster and more reliable interaction execution. Also, alignment provides a mechanism for more direct behavioral learning. We investigated three manners of alignment, individual on-the-fly alignment, pre-interaction alignment, and alignment with the intervention of a third party. This paper explains in some detail how alignment on-the-fly is realized using alignment policies. The features of the other two kinds of alignment are discussed, and future directions for research are pointed out.

We compare the performance of two connectionist models developed to account for some specific aspects of the decision making process in the Iterated Prisoner’s Dilemma Game. Both models are based on common recurrent network architecture. The first of them uses a backward-oriented reinforcement learning algorithm for learning to play the game while the second one makes its move decisions based on generated predictions about future games, moves and payoffs. Both models involve prediction of the opponent move and of the expected payoff and have an in-built autoassociator in their architecture aimed at more efficient payoff matrix representation. The results of the simulations show that the model with explicit anticipation about game outcomes could reproduce the experimentally observed dependency of the cooperation rate on the so-called cooperation index thus showing the importance of anticipation in modeling the actual decision making process in human participants. The role of the models’ building blocks and mechanisms is investigated and discussed. Comparisons with experiments with human participants are presented.

This paper presents an experimental study using two robots. In the experiment, the robots navigated through an area with or without obstacles and had the goal to shift places with each other. Four different approaches (random, reactive, planning, anticipation) were used during the experiment and the times to accomplish the task were compared. The results indicate that the ability to anticipate the behavior of the other robot can be advantageous. However, the results also clearly show that anticipatory and planned behavior are not always better than a purely reactive strategy.