-
Informational Embodiment: Computational role of information structure in codes and robots
Authors:
Alexandre Pitti,
Kohei Nakajima,
Yasuo Kuniyoshi
Abstract:
The body morphology plays an important role in the way information is perceived and processed by an agent. We address an information theory (IT) account on how the precision of sensors, the accuracy of motors, their placement, the body geometry, shape the information structure in robots and computational codes. As an original idea, we envision the robot's body as a physical communication channel t…
▽ More
The body morphology plays an important role in the way information is perceived and processed by an agent. We address an information theory (IT) account on how the precision of sensors, the accuracy of motors, their placement, the body geometry, shape the information structure in robots and computational codes. As an original idea, we envision the robot's body as a physical communication channel through which information is conveyed, in and out, despite intrinsic noise and material limitations. Following this, entropy, a measure of information and uncertainty, can be used to maximize the efficiency of robot design and of algorithmic codes per se. This is known as the principle of Entropy Maximization (PEM) introduced in biology by Barlow in 1969. The Shannon's source coding theorem provides then a framework to compare different types of bodies in terms of sensorimotor information. In line with PME, we introduce a special class of efficient codes used in IT that reached the Shannon limits in terms of information capacity for error correction and robustness against noise, and parsimony. These efficient codes, which exploit insightfully quantization and randomness, permit to deal with uncertainty, redundancy and compacity. These features can be used for perception and control in intelligent systems. In various examples and closing discussions, we reflect on the broader implications of our framework that we called Informational Embodiment to motor theory and bio-inspired robotics, touching upon concepts like motor synergies, reservoir computing, and morphological computation. These insights can contribute to a deeper understanding of how information theory intersects with the embodiment of intelligence in both natural and artificial systems.
△ Less
Submitted 23 August, 2024;
originally announced August 2024.
-
On the Relationship Between Variational Inference and Auto-Associative Memory
Authors:
Louis Annabi,
Alexandre Pitti,
Mathias Quoy
Abstract:
In this article, we propose a variational inference formulation of auto-associative memories, allowing us to combine perceptual inference and memory retrieval into the same mathematical framework. In this formulation, the prior probability distribution onto latent representations is made memory dependent, thus pulling the inference process towards previously stored representations. We then study h…
▽ More
In this article, we propose a variational inference formulation of auto-associative memories, allowing us to combine perceptual inference and memory retrieval into the same mathematical framework. In this formulation, the prior probability distribution onto latent representations is made memory dependent, thus pulling the inference process towards previously stored representations. We then study how different neural network approaches to variational inference can be applied in this framework. We compare methods relying on amortized inference such as Variational Auto Encoders and methods relying on iterative inference such as Predictive Coding and suggest combining both approaches to design new auto-associative memory models. We evaluate the obtained algorithms on the CIFAR10 and CLEVR image datasets and compare them with other associative memory models such as Hopfield Networks, End-to-End Memory Networks and Neural Turing Machines.
△ Less
Submitted 14 October, 2022;
originally announced October 2022.
-
A Predictive Coding Account for Chaotic Itinerancy
Authors:
Louis Annabi,
Alexandre Pitti,
Mathias Quoy
Abstract:
As a phenomenon in dynamical systems allowing autonomous switching between stable behaviors, chaotic itinerancy has gained interest in neurorobotics research. In this study, we draw a connection between this phenomenon and the predictive coding theory by showing how a recurrent neural network implementing predictive coding can generate neural trajectories similar to chaotic itinerancy in the prese…
▽ More
As a phenomenon in dynamical systems allowing autonomous switching between stable behaviors, chaotic itinerancy has gained interest in neurorobotics research. In this study, we draw a connection between this phenomenon and the predictive coding theory by showing how a recurrent neural network implementing predictive coding can generate neural trajectories similar to chaotic itinerancy in the presence of input noise. We propose two scenarios generating random and past-independent attractor switching trajectories using our model.
△ Less
Submitted 16 June, 2021;
originally announced June 2021.
-
Bidirectional Interaction between Visual and Motor Generative Models using Predictive Coding and Active Inference
Authors:
Louis Annabi,
Alexandre Pitti,
Mathias Quoy
Abstract:
In this work, we build upon the Active Inference (AIF) and Predictive Coding (PC) frameworks to propose a neural architecture comprising a generative model for sensory prediction, and a distinct generative model for motor trajectories. We highlight how sequences of sensory predictions can act as rails guiding learning, control and online adaptation of motor trajectories. We furthermore inquire the…
▽ More
In this work, we build upon the Active Inference (AIF) and Predictive Coding (PC) frameworks to propose a neural architecture comprising a generative model for sensory prediction, and a distinct generative model for motor trajectories. We highlight how sequences of sensory predictions can act as rails guiding learning, control and online adaptation of motor trajectories. We furthermore inquire the effects of bidirectional interactions between the motor and the visual modules. The architecture is tested on the control of a simulated robotic arm learning to reproduce handwritten letters.
△ Less
Submitted 19 April, 2021;
originally announced April 2021.
-
Towards a Neural Model for Serial Order in Frontal Cortex: a Brain Theory from Memory Development to Higher-Level Cognition
Authors:
Alexandre Pitti,
Mathias Quoy,
Catherine Lavandier,
Sofiane Boucenna,
Wassim Swaileh,
Claudio Weidmann
Abstract:
In order to keep trace of information and grow up, the infant brain has to resolve the problem about where old information is located and how to index new ones. We propose that the immature prefrontal cortex (PFC) use its primary functionality of detecting hierarchical patterns in temporal signals as a second purpose to organize the spatial ordering of the cortical networks in the developing brain…
▽ More
In order to keep trace of information and grow up, the infant brain has to resolve the problem about where old information is located and how to index new ones. We propose that the immature prefrontal cortex (PFC) use its primary functionality of detecting hierarchical patterns in temporal signals as a second purpose to organize the spatial ordering of the cortical networks in the developing brain itself. Our hypothesis is that the PFC detects the hierarchical structure in temporal sequences in the form of ordinal patterns and use them to index information hierarchically in different parts of the brain. Henceforth, we propose that this mechanism for detecting patterns participates in the ordinal organization development of the brain itself; i.e., the bootstrapping of the connectome. By doing so, it gives the tools to the language-ready brain for manipulating abstract knowledge and planning temporally ordered information; i.e., the emergence of symbolic thinking and language. We will review neural models that can support such mechanisms and propose new ones. We will confront then our ideas with evidence from developmental, behavioral and brain results and make some hypotheses, for instance, on the construction of the mirror neuron system, on embodied cognition, and on the capacity of learning-to-learn.
△ Less
Submitted 30 December, 2021; v1 submitted 22 May, 2020;
originally announced May 2020.
-
Autonomous learning and chaining of motor primitives using the Free Energy Principle
Authors:
Louis Annabi,
Alexandre Pitti,
Mathias Quoy
Abstract:
In this article, we apply the Free-Energy Principle to the question of motor primitives learning. An echo-state network is used to generate motor trajectories. We combine this network with a perception module and a controller that can influence its dynamics. This new compound network permits the autonomous learning of a repertoire of motor trajectories. To evaluate the repertoires built with our m…
▽ More
In this article, we apply the Free-Energy Principle to the question of motor primitives learning. An echo-state network is used to generate motor trajectories. We combine this network with a perception module and a controller that can influence its dynamics. This new compound network permits the autonomous learning of a repertoire of motor trajectories. To evaluate the repertoires built with our method, we exploit them in a handwriting task where primitives are chained to produce long-range sequences.
△ Less
Submitted 11 May, 2020;
originally announced May 2020.
-
Ideas from Developmental Robotics and Embodied AI on the Questions of Ethics in Robots
Authors:
Alexandre Pitti
Abstract:
Advances in Artificial Intelligence and robotics are currently questioning theethical framework of their applications to deal with potential drifts, as well as the way inwhich these algorithms learn because they will have a strong impact on the behavior ofrobots and the type of robots. interactions with people. We would like to highlight someprinciples and ideas from cognitive neuroscience and dev…
▽ More
Advances in Artificial Intelligence and robotics are currently questioning theethical framework of their applications to deal with potential drifts, as well as the way inwhich these algorithms learn because they will have a strong impact on the behavior ofrobots and the type of robots. interactions with people. We would like to highlight someprinciples and ideas from cognitive neuroscience and development sciences based on theimportance of the body for intelligence, contrary to the theory of the all-brain or all-algorithm, to represent the world and interacting with others, and their current applicationsin embodied AI and developmental robotics to propose models of architectures andmechanisms for agency, representation of the body, recognition of the intention of others,predictive coding, active inference, the role of feedback and error, imitation, artificialcuriosity and contextual learning. We will explain how these are important for the design ofautonomous systems and beyond what they can tell us for the ethics of systems.
△ Less
Submitted 20 March, 2018;
originally announced March 2018.