-
Safe Reinforcement Learning in a Simulated Robotic Arm
Authors:
Luka Kovač,
Igor Farkaš
Abstract:
Reinforcement learning (RL) agents need to explore their environments in order to learn optimal policies. In many environments and tasks, safety is of critical importance. The widespread use of simulators offers a number of advantages, including safe exploration which will be inevitable in cases when RL systems need to be trained directly in the physical environment (e.g. in human-robot interactio…
▽ More
Reinforcement learning (RL) agents need to explore their environments in order to learn optimal policies. In many environments and tasks, safety is of critical importance. The widespread use of simulators offers a number of advantages, including safe exploration which will be inevitable in cases when RL systems need to be trained directly in the physical environment (e.g. in human-robot interaction). The popular Safety Gym library offers three mobile agent types that can learn goal-directed tasks while considering various safety constraints. In this paper, we extend the applicability of safe RL algorithms by creating a customized environment with Panda robotic arm where Safety Gym algorithms can be tested. We performed pilot experiments with the popular PPO algorithm comparing the baseline with the constrained version and show that the constrained version is able to learn the equally good policy while better complying with safety constraints and taking longer training time as expected.
△ Less
Submitted 28 February, 2024; v1 submitted 28 November, 2023;
originally announced December 2023.
-
Appearance-based gaze estimation enhanced with synthetic images using deep neural networks
Authors:
Dmytro Herashchenko,
Igor Farkaš
Abstract:
Human eye gaze estimation is an important cognitive ingredient for successful human-robot interaction, enabling the robot to read and predict human behavior. We approach this problem using artificial neural networks and build a modular system estimating gaze from separately cropped eyes, taking advantage of existing well-functioning components for face detection (RetinaFace) and head pose estimati…
▽ More
Human eye gaze estimation is an important cognitive ingredient for successful human-robot interaction, enabling the robot to read and predict human behavior. We approach this problem using artificial neural networks and build a modular system estimating gaze from separately cropped eyes, taking advantage of existing well-functioning components for face detection (RetinaFace) and head pose estimation (6DRepNet). Our proposed method does not require any special hardware or infrared filters but uses a standard notebook-builtin RGB camera, as often approached with appearance-based methods. Using the MetaHuman tool, we also generated a large synthetic dataset of more than 57,000 human faces and made it publicly available. The inclusion of this dataset (with eye gaze and head pose information) on top of the standard Columbia Gaze dataset into training the model led to better accuracy with a mean average error below two degrees in eye pitch and yaw directions, which compares favourably to related methods. We also verified the feasibility of our model by its preliminary testing in real-world setting using the builtin 4K camera in NICO semi-humanoid robot's eye.
△ Less
Submitted 28 February, 2024; v1 submitted 23 November, 2023;
originally announced November 2023.
-
Robot at the Mirror: Learning to Imitate via Associating Self-supervised Models
Authors:
Andrej Lucny,
Kristina Malinovska,
Igor Farkas
Abstract:
We introduce an approach to building a custom model from ready-made self-supervised models via their associating instead of training and fine-tuning. We demonstrate it with an example of a humanoid robot looking at the mirror and learning to detect the 3D pose of its own body from the image it perceives. To build our model, we first obtain features from the visual input and the postures of the rob…
▽ More
We introduce an approach to building a custom model from ready-made self-supervised models via their associating instead of training and fine-tuning. We demonstrate it with an example of a humanoid robot looking at the mirror and learning to detect the 3D pose of its own body from the image it perceives. To build our model, we first obtain features from the visual input and the postures of the robot's body via models prepared before the robot's operation. Then, we map their corresponding latent spaces by a sample-efficient robot's self-exploration at the mirror. In this way, the robot builds the solicited 3D pose detector, which quality is immediately perfect on the acquired samples instead of obtaining the quality gradually. The mapping, which employs associating the pairs of feature vectors, is then implemented in the same way as the key-value mechanism of the famous transformer models. Finally, deploying our model for imitation to a simulated robot allows us to study, tune up, and systematically evaluate its hyperparameters without the involvement of the human counterpart, advancing our previous research.
△ Less
Submitted 26 February, 2024; v1 submitted 22 November, 2023;
originally announced November 2023.
-
From positional representation of numbers to positional representation of vectors
Authors:
Izabella Ingrid Farkas,
Edita Pelantová,
Milena Svobodová
Abstract:
To represent real $m$-dimensional vectors, a positional vector system given by a non-singular matrix $M \in \mathbb{Z}^{m \times m}$ and a digit set $\mathcal{D} \subset \mathbb{Z}^m$ is used. If $m = 1$, the system coincides with the well known numeration system used to represent real numbers. We study some properties of the vector systems which are transformable from the case $m = 1$ to higher d…
▽ More
To represent real $m$-dimensional vectors, a positional vector system given by a non-singular matrix $M \in \mathbb{Z}^{m \times m}$ and a digit set $\mathcal{D} \subset \mathbb{Z}^m$ is used. If $m = 1$, the system coincides with the well known numeration system used to represent real numbers. We study some properties of the vector systems which are transformable from the case $m = 1$ to higher dimensions. We focus on algorithm for parallel addition and on systems allowing an eventually periodic representation of vectors with rational coordinates.
△ Less
Submitted 17 March, 2023;
originally announced March 2023.
-
Self-supervised network distillation: an effective approach to exploration in sparse reward environments
Authors:
Matej Pecháč,
Michal Chovanec,
Igor Farkaš
Abstract:
Reinforcement learning can solve decision-making problems and train an agent to behave in an environment according to a predesigned reward function. However, such an approach becomes very problematic if the reward is too sparse and so the agent does not come across the reward during the environmental exploration. The solution to such a problem may be to equip the agent with an intrinsic motivation…
▽ More
Reinforcement learning can solve decision-making problems and train an agent to behave in an environment according to a predesigned reward function. However, such an approach becomes very problematic if the reward is too sparse and so the agent does not come across the reward during the environmental exploration. The solution to such a problem may be to equip the agent with an intrinsic motivation that will provide informed exploration during which the agent is likely to also encounter external reward. Novelty detection is one of the promising branches of intrinsic motivation research. We present Self-supervised Network Distillation (SND), a class of intrinsic motivation algorithms based on the distillation error as a novelty indicator, where the predictor model and the target model are both trained. We adapted three existing self-supervised methods for this purpose and experimentally tested them on a set of ten environments that are considered difficult to explore. The results show that our approach achieves faster growth and higher external reward for the same training time compared to the baseline models, which implies improved exploration in a very sparse reward environment. In addition, the analytical methods we applied provide valuable explanatory insights into our proposed models.
△ Less
Submitted 10 June, 2024; v1 submitted 22 February, 2023;
originally announced February 2023.
-
Examining the Proximity of Adversarial Examples to Class Manifolds in Deep Networks
Authors:
Štefan Pócoš,
Iveta Bečková,
Igor Farkaš
Abstract:
Deep neural networks achieve remarkable performance in multiple fields. However, after proper training they suffer from an inherent vulnerability against adversarial examples (AEs). In this work we shed light on inner representations of the AEs by analysing their activations on the hidden layers. We test various types of AEs, each crafted using a specific norm constraint, which affects their visua…
▽ More
Deep neural networks achieve remarkable performance in multiple fields. However, after proper training they suffer from an inherent vulnerability against adversarial examples (AEs). In this work we shed light on inner representations of the AEs by analysing their activations on the hidden layers. We test various types of AEs, each crafted using a specific norm constraint, which affects their visual appearance and eventually their behavior in the trained networks. Our results in image classification tasks (MNIST and CIFAR-10) reveal qualitative differences between the individual types of AEs, when comparing their proximity to the class-specific manifolds on the inner representations. We propose two methods that can be used to compare the distances to class-specific manifolds, regardless of the changing dimensions throughout the network. Using these methods, we consistently confirm that some of the adversarials do not necessarily leave the proximity of the manifold of the correct class, not even in the last hidden layer of the neural network. Next, using UMAP visualisation technique, we project the class activations to 2D space. The results indicate that the activations of the individual AEs are entangled with the activations of the test set. This, however, does not hold for a group of crafted inputs called the rubbish class. We also confirm the entanglement of adversarials with the test set numerically using the soft nearest neighbour loss.
△ Less
Submitted 12 April, 2022;
originally announced April 2022.
-
Quantifying the changing role of past publications
Authors:
Katalin Orosz,
Illes J. Farkas,
Peter Pollner
Abstract:
Our current societies increasingly rely on electronic repositories of collective knowledge. An archetype of these databases is the Web of Science (WoS) that stores scientific publications. In contrast to several other forms of knowledge -- e.g., Wikipedia articles -- a scientific paper does not change after its "birth". Nonetheless, from the moment a paper is published it exists within the evolvin…
▽ More
Our current societies increasingly rely on electronic repositories of collective knowledge. An archetype of these databases is the Web of Science (WoS) that stores scientific publications. In contrast to several other forms of knowledge -- e.g., Wikipedia articles -- a scientific paper does not change after its "birth". Nonetheless, from the moment a paper is published it exists within the evolving web of other papers, thus, its actual meaning to the reader changes. To track how scientific ideas (represented by groups of scientific papers) appear and evolve, we apply a novel combination of algorithms explicitly allowing for papers to change their groups. We (i) identify the overlapping clusters of the undirected yearly co-citation networks of the WoS (1975-2008) and (ii) match these yearly clusters (groups) to form group timelines. After visualizing the longest lived groups of the entire data set we assign topic labels to the groups. We find that in the entire Web of Science multidisciplinarity is clearly over-represented among cutting edge ideas. In addition, we provide detailed examples for papers that (i) change their topic labels and (ii) move between groups.
△ Less
Submitted 2 May, 2016;
originally announced May 2016.
-
Scientometrics: Untangling the topics
Authors:
Adam Szanto-Varnagy,
Peter Pollner,
Tamas Vicsek,
Illes J. Farkas
Abstract:
Measuring science is based on comparing articles to similar others. However, keyword-based groups of thematically similar articles are dominantly small. These small sizes keep the statistical errors of comparisons high. With the growing availability of bibliographic data such statistical errors can be reduced by merging methods of thematic grouping, citation networks and keyword co-usage.
Measuring science is based on comparing articles to similar others. However, keyword-based groups of thematically similar articles are dominantly small. These small sizes keep the statistical errors of comparisons high. With the growing availability of bibliographic data such statistical errors can be reduced by merging methods of thematic grouping, citation networks and keyword co-usage.
△ Less
Submitted 18 April, 2014; v1 submitted 10 March, 2014;
originally announced March 2014.
