Showing 1–41 of 41 results for author: Mistry, M

A Comparative Study of Sentence Embedding Models for Assessing Semantic Variation

Authors: Deven M. Mistry, Ali A. Minai

Abstract: Analyzing the pattern of semantic variation in long real-world texts such as books or transcripts is interesting from the stylistic, cognitive, and linguistic perspectives. It is also useful for applications such as text segmentation, document summarization, and detection of semantic novelty. The recent emergence of several vector-space methods for sentence embedding has made such analysis feasibl… ▽ More Analyzing the pattern of semantic variation in long real-world texts such as books or transcripts is interesting from the stylistic, cognitive, and linguistic perspectives. It is also useful for applications such as text segmentation, document summarization, and detection of semantic novelty. The recent emergence of several vector-space methods for sentence embedding has made such analysis feasible. However, this raises the issue of how consistent and meaningful the semantic representations produced by various methods are in themselves. In this paper, we compare several recent sentence embedding methods via time-series of semantic similarity between successive sentences and matrices of pairwise sentence similarity for multiple books of literature. In contrast to previous work using target tasks and curated datasets to compare sentence embedding methods, our approach provides an evaluation of the methods 'in the wild'. We find that most of the sentence embedding methods considered do infer highly correlated patterns of semantic similarity in a given document, but show interesting differences. △ Less

Submitted 8 August, 2023; originally announced August 2023.

Comments: 12 pages, 6 figures, Accepted for publication in the Proceedings of the 2023 International Conference on Artificial Neural Networks, Heraklion,, Greece, September 26-29, 2023

Neural Lyapunov and Optimal Control

Authors: Daniel Layeghi, Steve Tonneau, Michael Mistry

Abstract: Despite impressive results, reinforcement learning (RL) suffers from slow convergence and requires a large variety of tuning strategies. In this paper, we investigate the ability of RL algorithms on simple continuous control tasks. We show that without reward and environment tuning, RL suffers from poor convergence. In turn, we introduce an optimal control (OC) theoretic learning-based method that… ▽ More Despite impressive results, reinforcement learning (RL) suffers from slow convergence and requires a large variety of tuning strategies. In this paper, we investigate the ability of RL algorithms on simple continuous control tasks. We show that without reward and environment tuning, RL suffers from poor convergence. In turn, we introduce an optimal control (OC) theoretic learning-based method that can solve the same problems robustly with simple parsimonious costs. We use the Hamilton-Jacobi-Bellman (HJB) and first-order gradients to learn optimal time-varying value functions and therefore, policies. We show the relaxation of our objective results in time-varying Lyapunov functions, further verifying our approach by providing guarantees over a compact set of initial conditions. We compare our method to Soft Actor Critic (SAC) and Proximal Policy Optimisation (PPO). In this comparison, we solve all tasks, we never underperform in task cost and we show that at the point of our convergence, we outperform SAC and PPO in the best case by 4 and 2 orders of magnitude. △ Less

Submitted 15 February, 2024; v1 submitted 24 May, 2023; originally announced May 2023.

Single-shot Foothold Selection and Constraint Evaluation for Quadruped Locomotion

Authors: D. Belter, J. Bednarek, H. -C. Lin, G. Xin, M. Mistry

Abstract: In this paper, we propose a method for selecting the optimal footholds for legged systems. The goal of the proposed method is to find the best foothold for the swing leg on a local elevation map. We apply the Convolutional Neural Network to learn the relationship between the local elevation map and the quality of potential footholds. The proposed network evaluates the geometrical characteristics o… ▽ More In this paper, we propose a method for selecting the optimal footholds for legged systems. The goal of the proposed method is to find the best foothold for the swing leg on a local elevation map. We apply the Convolutional Neural Network to learn the relationship between the local elevation map and the quality of potential footholds. The proposed network evaluates the geometrical characteristics of each cell on the elevation map, checks kinematic constraints and collisions. During execution time, the controller obtains the qualitative measurement of each potential foothold from the neural model. This method allows to evaluate hundreds of potential footholds and check multiple constraints in a single step which takes 10~ms on a standard computer without GPGPU. The experiments were carried out on a quadruped robot walking over rough terrain in both simulation and real robotic platforms. △ Less

Submitted 1 December, 2022; originally announced December 2022.

Journal ref: IEEE, Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), 2019, pp. 7441-7447

Achieving Dexterous Bidirectional Interaction in Uncertain Conditions for Medical Robotics

Authors: Carlo Tiseo, Quentin Rouxel, Martin Asenov, Keyhan Kouhkiloui Babarahmati, Subramanian Ramamoorthy, Zhibin Li, Michael Mistry

Abstract: Medical robotics can help improve and extend the reach of healthcare services. A major challenge for medical robots is the complex physical interaction between the robot and the patients which is required to be safe. This work presents the preliminary evaluation of a recently introduced control architecture based on the Fractal Impedance Control (FIC) in medical applications. The deployed FIC arch… ▽ More Medical robotics can help improve and extend the reach of healthcare services. A major challenge for medical robots is the complex physical interaction between the robot and the patients which is required to be safe. This work presents the preliminary evaluation of a recently introduced control architecture based on the Fractal Impedance Control (FIC) in medical applications. The deployed FIC architecture is robust to delay between the master and the replica robots. It can switch online between an admittance and impedance behaviour, and it is robust to interaction with unstructured environments. Our experiments analyse three scenarios: teleoperated surgery, rehabilitation, and remote ultrasound scan. The experiments did not require any adjustment of the robot tuning, which is essential in medical applications where the operators do not have an engineering background required to tune the controller. Our results show that is possible to teleoperate the robot to cut using a scalpel, do an ultrasound scan, and perform remote occupational therapy. However, our experiments also highlighted the need for a better robots embodiment to precisely control the system in 3D dynamic tasks. △ Less

Submitted 20 June, 2022; originally announced June 2022.

Comments: video: https://youtu.be/G5NfFbh_ULg

Collaborative Bimanual Manipulation Using Optimal Motion Adaptation and Interaction Control

Authors: Ruoshi Wen, Quentin Rouxel, Michael Mistry, Zhibin Li, Carlo Tiseo

Abstract: This work developed collaborative bimanual manipulation for reliable and safe human-robot collaboration, which allows remote and local human operators to work interactively for bimanual tasks. We proposed an optimal motion adaptation to retarget arbitrary commands from multiple human operators into feasible control references. The collaborative manipulation framework has three main modules: (1) co… ▽ More This work developed collaborative bimanual manipulation for reliable and safe human-robot collaboration, which allows remote and local human operators to work interactively for bimanual tasks. We proposed an optimal motion adaptation to retarget arbitrary commands from multiple human operators into feasible control references. The collaborative manipulation framework has three main modules: (1) contact force modulation for compliant physical interactions with objects via admittance control; (2) task-space sequential equilibrium and inverse kinematics optimization, which adapts interactive commands from multiple operators to feasible motions by satisfying the task constraints and physical limits of the robots; and (3) an interaction controller adopted from the fractal impedance control, which is robust to time delay and stable to superimpose multiple control efforts for generating desired joint torques and controlling the dual-arm robots. Extensive experiments demonstrated the capability of the collaborative bimanual framework, including (1) dual-arm teleoperation that adapts arbitrary infeasible commands that violate joint torque limits into continuous operations within safe boundaries, compared to failures without the proposed optimization; (2) robust maneuver of a stack of objects via physical interactions in presence of model inaccuracy; (3) collaborative multi-operator part assembly, and teleoperated industrial connector insertion, which validate the guaranteed stability of reliable human-robot co-manipulation. △ Less

Submitted 18 July, 2023; v1 submitted 1 June, 2022; originally announced June 2022.

Comments: in IEEE Robotics & Automation Magazine, 2023

Efficient Learning of Inverse Dynamics Models for Adaptive Computed Torque Control

Authors: David Jorge, Gabriella Pizzuto, Michael Mistry

Abstract: Modelling robot dynamics accurately is essential for control, motion optimisation and safe human-robot collaboration. Given the complexity of modern robotic systems, dynamics modelling remains non-trivial, mostly in the presence of compliant actuators, mechanical inaccuracies, friction and sensor noise. Recent efforts have focused on utilising data-driven methods such as Gaussian processes and neu… ▽ More Modelling robot dynamics accurately is essential for control, motion optimisation and safe human-robot collaboration. Given the complexity of modern robotic systems, dynamics modelling remains non-trivial, mostly in the presence of compliant actuators, mechanical inaccuracies, friction and sensor noise. Recent efforts have focused on utilising data-driven methods such as Gaussian processes and neural networks to overcome these challenges, as they are capable of capturing these dynamics without requiring extensive knowledge beforehand. While Gaussian processes have shown to be an effective method for learning robotic dynamics with the ability to also represent the uncertainty in the learned model through its variance, they come at a cost of cubic time complexity rather than linear, as is the case for deep neural networks. In this work, we leverage the use of deep kernel models, which combine the computational efficiency of deep learning with the non-parametric flexibility of kernel methods (Gaussian processes), with the overarching goal of realising an accurate probabilistic framework for uncertainty quantification. Through using the predicted variance, we adapt the feedback gains as more accurate models are learned, leading to low-gain control without compromising tracking accuracy. Using simulated and real data recorded from a seven degree-of-freedom robotic manipulator, we illustrate how using stochastic variational inference with deep kernel models increases compliance in the computed torque controller, and retains tracking accuracy. We empirically show how our model outperforms current state-of-the-art methods with prediction uncertainty for online inverse dynamics model learning, and solidify its adaptation and generalisation capabilities across different setups. △ Less

Submitted 10 May, 2022; originally announced May 2022.

Comments: Submitted to IEEE/RSJ International Conference on Intelligent Robots & Systems (IROS) 2022

Agile Maneuvers in Legged Robots: a Predictive Control Approach

Authors: Carlos Mastalli, Wolfgang Merkt, Guiyang Xin, Jaehyun Shim, Michael Mistry, Ioannis Havoutis, Sethu Vijayakumar

Abstract: Planning and execution of agile locomotion maneuvers have been a longstanding challenge in legged robotics. It requires to derive motion plans and local feedback policies in real-time to handle the nonholonomy of the kinetic momenta. To achieve so, we propose a hybrid predictive controller that considers the robot's actuation limits and full-body dynamics. It combines the feedback policies with ta… ▽ More Planning and execution of agile locomotion maneuvers have been a longstanding challenge in legged robotics. It requires to derive motion plans and local feedback policies in real-time to handle the nonholonomy of the kinetic momenta. To achieve so, we propose a hybrid predictive controller that considers the robot's actuation limits and full-body dynamics. It combines the feedback policies with tactile information to locally predict future actions. It converges within a few milliseconds thanks to a feasibility-driven approach. Our predictive controller enables ANYmal robots to generate agile maneuvers in realistic scenarios. A crucial element is to track the local feedback policies as, in contrast to whole-body control, they achieve the desired angular momentum. To the best of our knowledge, our predictive controller is the first to handle actuation limits, generate agile locomotion maneuvers, and execute optimal feedback policies for low level torque control without the use of a separate whole-body controller. △ Less

Submitted 18 July, 2022; v1 submitted 14 March, 2022; originally announced March 2022.

Comments: 20 pages, 16 figures

What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural Networks

Authors: Kevin Sebastian Luck, Roberto Calandra, Michael Mistry

Abstract: The co-adaptation of robot morphology and behaviour becomes increasingly important with the advent of fast 3D-manufacturing methods and efficient deep reinforcement learning algorithms. A major challenge for the application of co-adaptation methods to the real world is the simulation-to-reality-gap due to model and simulation inaccuracies. However, prior work focuses primarily on the study of evol… ▽ More The co-adaptation of robot morphology and behaviour becomes increasingly important with the advent of fast 3D-manufacturing methods and efficient deep reinforcement learning algorithms. A major challenge for the application of co-adaptation methods to the real world is the simulation-to-reality-gap due to model and simulation inaccuracies. However, prior work focuses primarily on the study of evolutionary adaptation of morphologies exploiting analytical models and (differentiable) simulators with large population sizes, neglecting the existence of the simulation-to-reality-gap and the cost of manufacturing cycles in the real world. This paper presents a new approach combining classic high-frequency deep neural networks with computational expensive Graph Neural Networks for the data-efficient co-adaptation of agents with varying numbers of degrees-of-freedom. Evaluations in simulation show that the new method can co-adapt agents within such a limited number of production cycles by efficiently combining design optimization with offline reinforcement learning, that it allows for the direct application to real-world co-adaptation tasks in future work △ Less

Submitted 3 November, 2021; originally announced November 2021.

Optimal Control via Combined Inference and Numerical Optimization

Authors: Daniel Layeghi, Steve Tonneau, Michael Mistry

Abstract: Derivative based optimization methods are efficient at solving optimal control problems near local optima. However, their ability to converge halts when derivative information vanishes. The inference approach to optimal control does not have strict requirements on the objective landscape. However, sampling, the primary tool for solving such problems, tends to be much slower in computation time. We… ▽ More Derivative based optimization methods are efficient at solving optimal control problems near local optima. However, their ability to converge halts when derivative information vanishes. The inference approach to optimal control does not have strict requirements on the objective landscape. However, sampling, the primary tool for solving such problems, tends to be much slower in computation time. We propose a new method that combines second order methods with inference. We utilise the Kullback Leibler (KL) control framework to formulate an inference problem that computes the optimal controls from an adaptive distribution approximating the solution of the second order method. Our method allows for combining simple convex and non convex cost functions. This simplifies the process of cost function design and leverages the strengths of both inference and second order optimization. We compare our method to Model Predictive Path Integral (MPPI) and iterative Linear Quadratic Regulator (iLQG), outperforming both in sample efficiency and quality on manipulation and obstacle avoidance tasks. △ Less

Submitted 28 February, 2022; v1 submitted 23 September, 2021; originally announced September 2021.

A Unified Model with Inertia Shaping for Highly Dynamic Jumps of Legged Robots

Authors: Ke Wang, Guiyang Xin, Songyan Xin, Michael Mistry, Sethu Vijayakumar, Petar Kormushev

Abstract: To achieve highly dynamic jumps of legged robots, it is essential to control the rotational dynamics of the robot. In this paper, we aim to improve the jumping performance by proposing a unified model for planning highly dynamic jumps that can approximately model the centroidal inertia. This model abstracts the robot as a single rigid body for the base and point masses for the legs. The model is c… ▽ More To achieve highly dynamic jumps of legged robots, it is essential to control the rotational dynamics of the robot. In this paper, we aim to improve the jumping performance by proposing a unified model for planning highly dynamic jumps that can approximately model the centroidal inertia. This model abstracts the robot as a single rigid body for the base and point masses for the legs. The model is called the Lump Leg Single Rigid Body Model (LL-SRBM) and can be used to plan motions for both bipedal and quadrupedal robots. By taking the effects of leg dynamics into account, LL-SRBM provides a computationally efficient way for the motion planner to change the centroidal inertia of the robot with various leg configurations. Concurrently, we propose a novel contact detection method by using the norm of the average spatial velocity. After the contact is detected, the controller is switched to force control to achieve a soft landing. Twisting jump and forward jump experiments on the bipedal robot SLIDER and quadrupedal robot ANYmal demonstrate the improved jump performance by actively changing the centroidal inertia. These experiments also show the generalization and the robustness of the integrated planning and control framework. △ Less

Submitted 9 September, 2021; originally announced September 2021.

Comments: 8 pages

Fine Manipulation and Dynamic Interaction in Haptic Teleoperation

Authors: Carlo Tiseo, Quentin Rouxel, Zhibin Li, Michael Mistry

Abstract: The teleoperation of robots enables remote intervention in distant and dangerous tasks without putting the operator in harm's way. However, remote operation faces fundamental challenges due to limits in communication delays. The proposed work improves the performances of teleoperation architecture based on Fractal Impedance Controller (FIC) by integrating into the haptic teleoperation pipeline a p… ▽ More The teleoperation of robots enables remote intervention in distant and dangerous tasks without putting the operator in harm's way. However, remote operation faces fundamental challenges due to limits in communication delays. The proposed work improves the performances of teleoperation architecture based on Fractal Impedance Controller (FIC) by integrating into the haptic teleoperation pipeline a postural optimisation that also accounts for the replica robots' physical limitations. This update improves dynamic interactions by trading off tracking accuracy to maintain the system within its power limits. Thus, allowing fine manipulation without renouncing the robustness of the FIC controller. Additionally, the proposed method allows an online trade-off between tracking the autonomous trajectory and executing the teleoperated command, allowing their safe superimposition. The validated experimental results show that the proposed method is robust to increased communication delays. Moreover, we demonstrated that the remote teleoperated robot remains stable and safe to interact with, even when the communication with the master side is abruptly interrupted. with, even when the communication with the master side is abruptly interrupted. △ Less

Submitted 5 May, 2022; v1 submitted 9 September, 2021; originally announced September 2021.

Robust Impedance Control for Dexterous Interaction Using Fractal Impedance Controller with IK-Optimisation

Authors: Carlo Tiseo, Quentin Rouxel, Zhibin Li, Michael Mistry

Abstract: Robust dynamic interactions are required to move robots in daily environments alongside humans. Optimisation and learning methods have been used to mimic and reproduce human movements. However, they are often not robust and their generalisation is limited. This work proposed a hierarchical control architecture for robot manipulators and provided capabilities of reproducing human-like motions durin… ▽ More Robust dynamic interactions are required to move robots in daily environments alongside humans. Optimisation and learning methods have been used to mimic and reproduce human movements. However, they are often not robust and their generalisation is limited. This work proposed a hierarchical control architecture for robot manipulators and provided capabilities of reproducing human-like motions during unknown interaction dynamics. Our results show that the reproduced end-effector trajectories can preserve the main characteristics of the initial human motion recorded via a motion capture system, and are robust against external perturbations. The data indicate that some detailed movements are hard to reproduce due to the physical limits of the hardware that cannot reach the same velocity recorded in human movements. Nevertheless, these technical problems can be addressed by using better hardware and our proposed algorithms can still be applied to produce imitated motions. △ Less

Submitted 20 June, 2022; v1 submitted 9 September, 2021; originally announced September 2021.

Journal ref: 2022 IEEE International Conference on Robotics and Automation (ICRA)

Robust and Dexterous Dual-arm Tele-Cooperation using Adaptable Impedance Control

Authors: Keyhan Kouhkiloui Babarahmati, Mohammadreza Kasaei, Carlo Tiseo, Michael Mistry, Sethu Vijayakumar

Abstract: In recent years, the need for robots to transition from isolated industrial tasks to shared environments, including human-robot collaboration and teleoperation, has become increasingly evident. Building on the foundation of Fractal Impedance Control (FIC) introduced in our previous work, this paper presents a novel extension to dual-arm tele-cooperation, leveraging the non-linear stiffness and pas… ▽ More In recent years, the need for robots to transition from isolated industrial tasks to shared environments, including human-robot collaboration and teleoperation, has become increasingly evident. Building on the foundation of Fractal Impedance Control (FIC) introduced in our previous work, this paper presents a novel extension to dual-arm tele-cooperation, leveraging the non-linear stiffness and passivity of FIC to adapt to diverse cooperative scenarios. Unlike traditional impedance controllers, our approach ensures stability without relying on energy tanks, as demonstrated in our prior research. In this paper, we further extend the FIC framework to bimanual operations, allowing for stable and smooth switching between different dynamic tasks without gain tuning. We also introduce a telemanipulation architecture that offers higher transparency and dexterity, addressing the challenges of signal latency and low-bandwidth communication. Through extensive experiments, we validate the robustness of our method and the results confirm the advantages of the FIC approach over traditional impedance controllers, showcasing its potential for applications in planetary exploration and other scenarios requiring dexterous telemanipulation. This paper's contributions include the seamless integration of FIC into multi-arm systems, the ability to perform robust interactions in highly variable environments, and the provision of a comprehensive comparison with competing approaches, thereby significantly enhancing the robustness and adaptability of robotic systems. △ Less

Submitted 29 February, 2024; v1 submitted 10 August, 2021; originally announced August 2021.

Geometrical Postural Optimisation of 7-DoF Limb-Like Manipulators

Authors: Carlo Tiseo, Sydney Rebecca Charitos, Michael Mistry

Abstract: Robots are moving towards applications in less structured environments, but their model-based controllers are challenged by the tasks' complexity and intrinsic environmental unpredictability. Studying biological motor control can provide insights into overcoming these limitations due to the high dexterity and stability observable in humans and animals. This work presents a geometrical solution to… ▽ More Robots are moving towards applications in less structured environments, but their model-based controllers are challenged by the tasks' complexity and intrinsic environmental unpredictability. Studying biological motor control can provide insights into overcoming these limitations due to the high dexterity and stability observable in humans and animals. This work presents a geometrical solution to the postural optimisation of 7-DoF limbs-like mechanisms, which are robust to singularities and computationally efficient. The theoretical formulation identified two separate decoupled optimisation strategies. The shoulder and elbow strategy align the plane of motion with the expected plane of motion and guarantee the reachability of the end-posture. The wrist strategy ensures the end-effector orientation, which is essential to retain manipulability when nearing a singular configuration. The numerical results confirmed the theoretical observations and allowed us to identify the effect of different grasp strategies on system manipulability. The geometrical method was numerically tested in thousands of configurations proving to be both robust and accurate. The tested scenarios include left and right arm postures, singular configurations, and walking scenarios. The proposed geometrical approach can find application in developing efficient and robust interaction controllers that could be applied in computational neuroscience and robotics. △ Less

Submitted 20 August, 2021; v1 submitted 6 July, 2021; originally announced July 2021.

Journal ref: 2022, Eng. Res. Express

HapFIC: An Adaptive Force/Position Controller for Safe Environment Interaction in Articulated Systems

Authors: Carlo Tiseo, Wolfgang Merkt, Keyhan Kouhkiloui Babarahmati, Wouter Wolfslag, Ioannis Havoutis, Sethu Vijayakumar, Michael Mistry

Abstract: Haptic interaction is essential for the dynamic dexterity of animals, which seamlessly switch from an impedance to an admittance behaviour using the force feedback from their proprioception. However, this ability is extremely challenging to reproduce in robots, especially when dealing with complex interaction dynamics, distributed contacts, and contact switching. Current model-based controllers re… ▽ More Haptic interaction is essential for the dynamic dexterity of animals, which seamlessly switch from an impedance to an admittance behaviour using the force feedback from their proprioception. However, this ability is extremely challenging to reproduce in robots, especially when dealing with complex interaction dynamics, distributed contacts, and contact switching. Current model-based controllers require accurate interaction modelling to account for contacts and stabilise the interaction. In this manuscript, we propose an adaptive force/position controller that exploits the fractal impedance controller's passivity and non-linearity to execute a finite search algorithm using the force feedback signal from the sensor at the end-effector. The method is computationally inexpensive, opening the possibility to deal with distributed contacts in the future. We evaluated the architecture in physics simulation and showed that the controller can robustly control the interaction with objects of different dynamics without violating the maximum allowable target forces or causing numerical instability even for very rigid objects. The proposed controller can also autonomously deal with contact switching and may find application in multiple fields such as legged locomotion, rehabilitation and assistive robotics. △ Less

Submitted 20 July, 2021; v1 submitted 20 June, 2021; originally announced June 2021.

Comments: in IEEE Transactions on Neural Systems and Rehabilitation Engineering. Video: https://youtu.be/3FsVDZOIR1k

Exploiting Spherical Projections To Generate Human-Like Wrist Pointing Movements

Authors: Carlo Tiseo, Sydney Rebecca Charitos, Michael Mistry

Abstract: The mechanism behind the generation of human movements is of great interest in many fields (e.g. robotics and neuroscience) to improve therapies and technologies. Optimal Feedback Control (OFC) and Passive Motion Paradigm (PMP) are currently two leading theories capable of effectively producing human-like motions, but they require solving nonlinear inverse problems to find a solution. The main ben… ▽ More The mechanism behind the generation of human movements is of great interest in many fields (e.g. robotics and neuroscience) to improve therapies and technologies. Optimal Feedback Control (OFC) and Passive Motion Paradigm (PMP) are currently two leading theories capable of effectively producing human-like motions, but they require solving nonlinear inverse problems to find a solution. The main benefit of using PMP is the possibility of generating path-independent movements consistent with the stereotypical behaviour observed in humans, while the equivalent OFC formulation is path-dependent. Our results demonstrate how the path-independent behaviour observed for the wrist pointing task can be explained by spherical projections of the planar tasks. The combination of the projections with the fractal impedance controller eliminates the nonlinear inverse problem, which reduces the computational cost compared to previous methodologies. The motion exploits a recently proposed PMP architecture that replaces the nonlinear inverse optimisation with a nonlinear anisotropic stiffness impedance profile generated by the Fractal Impedance Controller, reducing the computational cost and not requiring a task-dependent optimisation. △ Less

Submitted 28 July, 2021; v1 submitted 8 March, 2021; originally announced March 2021.

Journal ref: In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

Knowledge Graphs Evolution and Preservation -- A Technical Report from ISWS 2019

Authors: Nacira Abbas, Kholoud Alghamdi, Mortaza Alinam, Francesca Alloatti, Glenda Amaral, Claudia d'Amato, Luigi Asprino, Martin Beno, Felix Bensmann, Russa Biswas, Ling Cai, Riley Capshaw, Valentina Anita Carriero, Irene Celino, Amine Dadoun, Stefano De Giorgis, Harm Delva, John Domingue, Michel Dumontier, Vincent Emonet, Marieke van Erp, Paola Espinoza Arias, Omaima Fallatah, Sebastián Ferrada, Marc Gallofré Ocaña , et al. (49 additional authors not shown)

Abstract: One of the grand challenges discussed during the Dagstuhl Seminar "Knowledge Graphs: New Directions for Knowledge Representation on the Semantic Web" and described in its report is that of a: "Public FAIR Knowledge Graph of Everything: We increasingly see the creation of knowledge graphs that capture information about the entirety of a class of entities. [...] This grand challenge extends this fur… ▽ More One of the grand challenges discussed during the Dagstuhl Seminar "Knowledge Graphs: New Directions for Knowledge Representation on the Semantic Web" and described in its report is that of a: "Public FAIR Knowledge Graph of Everything: We increasingly see the creation of knowledge graphs that capture information about the entirety of a class of entities. [...] This grand challenge extends this further by asking if we can create a knowledge graph of "everything" ranging from common sense concepts to location based entities. This knowledge graph should be "open to the public" in a FAIR manner democratizing this mass amount of knowledge." Although linked open data (LOD) is one knowledge graph, it is the closest realisation (and probably the only one) to a public FAIR Knowledge Graph (KG) of everything. Surely, LOD provides a unique testbed for experimenting and evaluating research hypotheses on open and FAIR KG. One of the most neglected FAIR issues about KGs is their ongoing evolution and long term preservation. We want to investigate this problem, that is to understand what preserving and supporting the evolution of KGs means and how these problems can be addressed. Clearly, the problem can be approached from different perspectives and may require the development of different approaches, including new theories, ontologies, metrics, strategies, procedures, etc. This document reports a collaborative effort performed by 9 teams of students, each guided by a senior researcher as their mentor, attending the International Semantic Web Research School (ISWS 2019). Each team provides a different perspective to the problem of knowledge graph evolution substantiated by a set of research questions as the main subject of their investigation. In addition, they provide their working definition for KG preservation and evolution. △ Less

Submitted 22 December, 2020; originally announced December 2020.

Theoretical Evidence Supporting Harmonic Reaching Trajectories

Authors: Carlo Tiseo, Sydney Rebecca Charitos, Michael Mistry

Abstract: Minimum Jerk trajectories have been long thought to be the reference trajectories for human movements due to their impressive similarity with human movements. Nevertheless, minimum jerk trajectories are not the only choice for $C^\infty$ (i.e., smooth) functions. For example, harmonic trajectories are smooth functions that can be superimposed to describe the evolution of physical systems. This pap… ▽ More Minimum Jerk trajectories have been long thought to be the reference trajectories for human movements due to their impressive similarity with human movements. Nevertheless, minimum jerk trajectories are not the only choice for $C^\infty$ (i.e., smooth) functions. For example, harmonic trajectories are smooth functions that can be superimposed to describe the evolution of physical systems. This paper analyses the possibility that motor control plans using harmonic trajectories, will be experimentally observed to have a minimum jerk likeness due to control signals being transported through the Central Nervous System (CNS) and muscle-skeletal system. We tested our theory on a 3-link arm simulation using a recently developed planner that we reformulated into a motor control architecture, inspired by the passive motion paradigm. The arm performed 100 movements, reaching for each target defined by the clock experiment. We analysed the shape of the trajectory planned in the CNS and executed in the physical simulator. We observed that even under ideal conditions (i.e., absence of delays and noise) the executed trajectories are similar to a minimum jerk trajectory; thus, supporting the thesis that the human brain might plan harmonic trajectories. △ Less

Submitted 4 March, 2021; v1 submitted 1 December, 2020; originally announced December 2020.

Journal ref: 2021 10th International IEEE/EMBS Conference on Neural Engineering (NER)

Iterative Semi-parametric Dynamics Model Learning For Autonomous Racing

Authors: Ignat Georgiev, Christoforos Chatzikomis, Timo Völkl, Joshua Smith, Michael Mistry

Abstract: Accurately modeling robot dynamics is crucial to safe and efficient motion control. In this paper, we develop and apply an iterative learning semi-parametric model, with a neural network, to the task of autonomous racing with a Model Predictive Controller (MPC). We present a novel non-linear semi-parametric dynamics model where we represent the known dynamics with a parametric model, and a neural… ▽ More Accurately modeling robot dynamics is crucial to safe and efficient motion control. In this paper, we develop and apply an iterative learning semi-parametric model, with a neural network, to the task of autonomous racing with a Model Predictive Controller (MPC). We present a novel non-linear semi-parametric dynamics model where we represent the known dynamics with a parametric model, and a neural network captures the unknown dynamics. We show that our model can learn more accurately than a purely parametric model and generalize better than a purely non-parametric model, making it ideal for real-world applications where collecting data from the full state space is not feasible. We present a system where the model is bootstrapped on pre-recorded data and then updated iteratively at run time. Then we apply our iterative learning approach to the simulated problem of autonomous racing and show that it can safely adapt to modified dynamics online and even achieve better performance than models trained on data from manual driving. △ Less

Submitted 17 November, 2020; originally announced November 2020.

Comments: Accepted at 4th Conference on Robot Learning (CoRL 2020)

ACM Class: I.2.9; I.2.6

A Passive Navigation Planning Algorithm for Collision-free Control of Mobile Robots

Authors: Carlo Tiseo, Vladimir Ivan, Wolfgang Merkt, Ioannis Havoutis, Michael Mistry, Sethu Vijayakumar

Abstract: Path planning and collision avoidance are challenging in complex and highly variable environments due to the limited horizon of events. In literature, there are multiple model- and learning-based approaches that require significant computational resources to be effectively deployed and they may have limited generality. We propose a planning algorithm based on a globally stable passive controller t… ▽ More Path planning and collision avoidance are challenging in complex and highly variable environments due to the limited horizon of events. In literature, there are multiple model- and learning-based approaches that require significant computational resources to be effectively deployed and they may have limited generality. We propose a planning algorithm based on a globally stable passive controller that can plan smooth trajectories using limited computational resources in challenging environmental conditions. The architecture combines the recently proposed fractal impedance controller with elastic bands and regions of finite time invariance. As the method is based on an impedance controller, it can also be used directly as a force/torque controller. We validated our method in simulation to analyse the ability of interactive navigation in challenging concave domains via the issuing of via-points, and its robustness to low bandwidth feedback. A swarm simulation using 11 agents validated the scalability of the proposed method. We have performed hardware experiments on a holonomic wheeled platform validating smoothness and robustness of interaction with dynamic agents (i.e., humans and robots). The computational complexity of the proposed local planner enables deployment with low-power micro-controllers lowering the energy consumption compared to other methods that rely upon numeric optimisation. △ Less

Submitted 20 June, 2022; v1 submitted 31 October, 2020; originally announced November 2020.

Journal ref: 2021 IEEE International Conference on Robotics and Automation (ICRA)

Robust Footstep Planning and LQR Control for Dynamic Quadrupedal Locomotion

Authors: Guiyang Xin, Songyan Xin, Oguzhan Cebe, Mathew Jose Pollayil, Franco Angelini, Manolo Garabini, Sethu Vijayakumar, Michael Mistry

Abstract: In this paper, we aim to improve the robustness of dynamic quadrupedal locomotion through two aspects: 1) fast model predictive foothold planning, and 2) applying LQR to projected inverse dynamic control for robust motion tracking. In our proposed planning and control framework, foothold plans are updated at 400 Hz considering the current robot state and an LQR controller generates optimal feedbac… ▽ More In this paper, we aim to improve the robustness of dynamic quadrupedal locomotion through two aspects: 1) fast model predictive foothold planning, and 2) applying LQR to projected inverse dynamic control for robust motion tracking. In our proposed planning and control framework, foothold plans are updated at 400 Hz considering the current robot state and an LQR controller generates optimal feedback gains for motion tracking. The LQR optimal gain matrix with non-zero off-diagonal elements leverages the coupling of dynamics to compensate for system underactuation. Meanwhile, the projected inverse dynamic control complements the LQR to satisfy inequality constraints. In addition to these contributions, we show robustness of our control framework to unmodeled adaptive feet. Experiments on the quadruped ANYmal demonstrate the effectiveness of the proposed method for robust dynamic locomotion given external disturbances and environmental uncertainties. △ Less

Submitted 13 March, 2021; v1 submitted 23 October, 2020; originally announced October 2020.

Online Dynamic Trajectory Optimization and Control for a Quadruped Robot

Authors: Oguzhan Cebe, Carlo Tiseo, Guiyang Xin, Hsiu-chin Lin, Joshua Smith, Michael Mistry

Abstract: Legged robot locomotion requires the planning of stable reference trajectories, especially while traversing uneven terrain. The proposed trajectory optimization framework is capable of generating dynamically stable base and footstep trajectories for multiple steps. The locomotion task can be defined with contact locations, base motion or both, making the algorithm suitable for multiple scenarios (… ▽ More Legged robot locomotion requires the planning of stable reference trajectories, especially while traversing uneven terrain. The proposed trajectory optimization framework is capable of generating dynamically stable base and footstep trajectories for multiple steps. The locomotion task can be defined with contact locations, base motion or both, making the algorithm suitable for multiple scenarios (e.g., presence of moving obstacles). The planner uses a simplified momentum-based task space model for the robot dynamics, allowing computation times that are fast enough for online replanning.This fast planning capabilitiy also enables the quadruped to accommodate for drift and environmental changes. The algorithm is tested on simulation and a real robot across multiple scenarios, which includes uneven terrain, stairs and moving obstacles. The results show that the planner is capable of generating stable trajectories in the real robot even when a box of 15 cm height is placed in front of its path at the last moment. △ Less

Submitted 23 March, 2021; v1 submitted 28 August, 2020; originally announced August 2020.

Comments: 7 pages, 10 figures, for video see https://bit.ly/3gyccU6

Variable Autonomy of Whole-body Control for Inspection and Intervention in Industrial Environments using Legged Robots

Authors: Guiyang Xin, Carlo Tiseo, Wouter Wolfslag, Joshua Smith, Oguzhan Cebe, Zhibin Li, Sethu Vijayakumar, Michael Mistry

Abstract: The deployment of robots in industrial and civil scenarios is a viable solution to protect operators from danger and hazards. Shared autonomy is paramount to enable remote control of complex systems such as legged robots, allowing the operator to focus on the essential tasks instead of overly detailed execution. To realize this, we propose a comprehensive control framework for inspection and inter… ▽ More The deployment of robots in industrial and civil scenarios is a viable solution to protect operators from danger and hazards. Shared autonomy is paramount to enable remote control of complex systems such as legged robots, allowing the operator to focus on the essential tasks instead of overly detailed execution. To realize this, we propose a comprehensive control framework for inspection and intervention using a legged robot and validate the integration of multiple loco-manipulation algorithms optimised for improving the remote operation. The proposed control offers 3 operation modes: fully automated, semi-autonomous, and the haptic interface receiving onsite physical interaction for assisting teleoperation. Our contribution is the design of a QP-based semi-analytical whole-body control, which is the key to the various task completion subject to internal and external constraints. We demonstrate the versatility of the whole-body control in terms of decoupling tasks, singularity tolerance and constraint satisfaction. We deploy our solution in field trials and evaluate in an emergency setting by an E-stop while the robot is clearing road barriers and traversing difficult terrains. △ Less

Submitted 23 October, 2020; v1 submitted 6 April, 2020; originally announced April 2020.

ENTMOOT: A Framework for Optimization over Ensemble Tree Models

Authors: Alexander Thebelt, Jan Kronqvist, Miten Mistry, Robert M. Lee, Nathan Sudermann-Merx, Ruth Misener

Abstract: Gradient boosted trees and other regression tree models perform well in a wide range of real-world, industrial applications. These tree models (i) offer insight into important prediction features, (ii) effectively manage sparse data, and (iii) have excellent prediction capabilities. Despite their advantages, they are generally unpopular for decision-making tasks and black-box optimization, which i… ▽ More Gradient boosted trees and other regression tree models perform well in a wide range of real-world, industrial applications. These tree models (i) offer insight into important prediction features, (ii) effectively manage sparse data, and (iii) have excellent prediction capabilities. Despite their advantages, they are generally unpopular for decision-making tasks and black-box optimization, which is due to their difficult-to optimize structure and the lack of a reliable uncertainty measure. ENTMOOT is our new framework for integrating (already trained) tree models into larger optimization problems. The contributions of ENTMOOT include: (i) explicitly introducing a reliable uncertainty measure that is compatible with tree models, (ii) solving the larger optimization problems that incorporate these uncertainty aware tree models, (iii) proving that the solutions are globally optimal, i.e. no better solution exists. In particular, we show how the ENTMOOT approach allows a simple integration of tree models into decision-making and black-box optimization, where it proves as a strong competitor to commonly-used frameworks. △ Less

Submitted 18 May, 2021; v1 submitted 10 March, 2020; originally announced March 2020.

Comments: 33 pages, 10 figures, 2 tables

Contact Surface Estimation via Haptic Perception

Authors: Hsiu-Chin Lin, Michael Mistry

Abstract: Legged systems need to optimize contact force in order to maintain contacts. For this, the controller needs to have the knowledge of the surface geometry and how slippery the terrain is. We can use a vision system to realize the terrain, but the accuracy of the vision system degrades in harsh weather, and it cannot visualize the terrain if it is covered with water or grass. Also, the degree of fri… ▽ More Legged systems need to optimize contact force in order to maintain contacts. For this, the controller needs to have the knowledge of the surface geometry and how slippery the terrain is. We can use a vision system to realize the terrain, but the accuracy of the vision system degrades in harsh weather, and it cannot visualize the terrain if it is covered with water or grass. Also, the degree of friction cannot be directly visualized. In this paper, we propose an online method to estimate the surface information via haptic exploration. We also introduce a probabilistic criterion to measure the quality of the estimation. The method is validated on both simulation and a real robot platform. △ Less

Submitted 4 March, 2020; originally announced March 2020.

Comments: Accepted for publications in IEEE International Conference on Robotics and Automation, 2020

Bio-mimetic Adaptive Force/Position Control Using Fractal Impedance

Authors: Carlo Tiseo, Wolfgang Merkt, Keyhan Kouhkiloui Babarahmati, Wouter Wolfslag, Sethu Vijayakumar, Michael Mistry

Abstract: The ability of animals to interact with complex dynamics is unmatched in robots. Especially important to the interaction performances is the online adaptation of body dynamics, which can be modeled as an impedance behaviour. However, the variable impedance controller still possesses a challenge in the current control frameworks due to the difficulties of retaining stability when adapting the contr… ▽ More The ability of animals to interact with complex dynamics is unmatched in robots. Especially important to the interaction performances is the online adaptation of body dynamics, which can be modeled as an impedance behaviour. However, the variable impedance controller still possesses a challenge in the current control frameworks due to the difficulties of retaining stability when adapting the controller gains. The fractal impedance controller has been recently proposed to solve this issue. However, it still has limitations such as sudden jumps in force when it starts to converge to the desired position and the lack of a force feedback loop. In this manuscript, two improvements are made to the control framework to solve these limitations. The force discontinuity has been addressed introducing a modulation of the impedance via a virtual antagonist that modulates the output force. The force tracking has been modeled after the parallel force/position controller architecture. In contrast to traditional methods, the fractal impedance controller enables the implementation of a search algorithm on the force feedback to adapt its behaviour on the external environment instead of on relying on \textit{a priori} knowledge of the external dynamics. Preliminary simulation results presented in this paper show the feasibility of the proposed approach, and it allows to evaluate the trade-off that needs to be made when relying on the proposed controller for interaction. In conclusion, the proposed method mimics the behaviour of an agonist/antagonist system adapting to unknown external dynamics, and it may find application in computational neuroscience, haptics, and interaction control. △ Less

Submitted 27 May, 2020; v1 submitted 3 March, 2020; originally announced March 2020.

Comments: \c{opyright} 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Journal ref: 2020 8th IEEE RAS/EMBS International Conference for Biomedical Robotics and Biomechatronics (BioRob)

Robust High-Transparency Haptic Exploration for Dexterous Telemanipulation

Authors: Keyhan Kouhkiloui Babarahmati, Carlo Tiseo, Quentin Rouxel, Zhibin Li, Michael Mistry

Abstract: Robotic teleoperation will allow us to perform complex manipulation tasks in dangerous or remote environments, such as needed for planetary exploration or nuclear decommissioning. This work proposes a novel telemanipulation architecture using a passive Fractal Impedance Controller (FIC), which does not depend upon an active viscous component for stability guarantees. Compared to a traditional impe… ▽ More Robotic teleoperation will allow us to perform complex manipulation tasks in dangerous or remote environments, such as needed for planetary exploration or nuclear decommissioning. This work proposes a novel telemanipulation architecture using a passive Fractal Impedance Controller (FIC), which does not depend upon an active viscous component for stability guarantees. Compared to a traditional impedance controller in ideal conditions (no delays and maximum communication bandwidth), our proposed method yields higher transparency in interaction and demonstrates superior dexterity and capability in our telemanipulation test scenarios. We also validate its performance with extreme delays up to 1 s and communication bandwidths as low as 10 Hz. All results validate a consistent stability when using the proposed controller in challenging conditions, regardless of operator expertise. △ Less

Submitted 22 June, 2022; v1 submitted 3 March, 2020; originally announced March 2020.

Journal ref: 2021 IEEE International Conference on Robotics and Automation (ICRA)

Adversarial Generation of Informative Trajectories for Dynamics System Identification

Authors: Marija Jegorova, Joshua Smith, Michael Mistry, Timothy Hospedales

Abstract: Dynamic System Identification approaches usually heavily rely on the evolutionary and gradient-based optimisation techniques to produce optimal excitation trajectories for determining the physical parameters of robot platforms. Current optimisation techniques tend to generate single trajectories. This is expensive, and intractable for longer trajectories, thus limiting their efficacy for system id… ▽ More Dynamic System Identification approaches usually heavily rely on the evolutionary and gradient-based optimisation techniques to produce optimal excitation trajectories for determining the physical parameters of robot platforms. Current optimisation techniques tend to generate single trajectories. This is expensive, and intractable for longer trajectories, thus limiting their efficacy for system identification. We propose to tackle this issue by using multiple shorter cyclic trajectories, which can be generated in parallel, and subsequently combined together to achieve the same effect as a longer trajectory. Crucially, we show how to scale this approach even further by increasing the generation speed and quality of the dataset through the use of generative adversarial network (GAN) based architectures to produce a large databases of valid and diverse excitation trajectories. To the best of our knowledge, this is the first robotics work to explore system identification with multiple cyclic trajectories and to develop GAN-based techniques for scaleably producing excitation trajectories that are diverse in both control parameter and inertial parameter spaces. We show that our approach dramatically accelerates trajectory optimisation, while simultaneously providing more accurate system identification than the conventional approach. △ Less

Submitted 23 September, 2020; v1 submitted 2 March, 2020; originally announced March 2020.

Comments: Accepted for publication in IEEE iROS 2020

Safe and Compliant Control of Redundant Robots Using Superimposition of Passive Task-Space Controllers

Authors: Carlo Tiseo, Wolfgang Merkt, Wouter Wolfslag, Sethu Vijayakumar, Michael Mistry

Abstract: Safe and compliant control of dynamic systems in interaction with the environment, e.g., in shared workspaces, continues to represent a major challenge. Mismatches in the dynamic model of the robots, numerical singularities, and the intrinsic environmental unpredictability are all contributing factors. Online optimization of impedance controllers has recently shown great promise in addressing this… ▽ More Safe and compliant control of dynamic systems in interaction with the environment, e.g., in shared workspaces, continues to represent a major challenge. Mismatches in the dynamic model of the robots, numerical singularities, and the intrinsic environmental unpredictability are all contributing factors. Online optimization of impedance controllers has recently shown great promise in addressing this challenge, however, their performance is not sufficiently robust to be deployed in challenging environments. This work proposes a compliant control method for redundant manipulators based on a superimposition of multiple passive task-space controllers in a hierarchy. Our control framework of passive controllers is inherently stable, numerically well-conditioned (as no matrix inversions are required), and computationally inexpensive (as no optimization is used). We leverage and introduce a novel stiffness profile for a recently proposed passive controller with smooth transitions between the divergence and convergence phases making it particularly suitable when multiple passive controllers are combined through superimposition. Our experimental results demonstrate that the proposed method achieves sub-centimeter tracking performance during demanding dynamic tasks with fast-changing references, while remaining safe to interact with and robust to singularities. he proposed framework achieves such results without knowledge of the robot dynamics and thanks to its passivity is intrinsically stable. The data further show that the robot can fully take advantage of the redundancy to maintain the primary task accuracy while compensating for unknown environmental interactions, which is not possible from current frameworks that require accurate contact information. △ Less

Submitted 21 September, 2020; v1 submitted 27 February, 2020; originally announced February 2020.

Comments: Nonlinear Dyn (2023)

Bounded haptic teleoperation of a quadruped robot's foot posture for sensing and manipulation

Authors: Guiyang Xin, Joshua Smith, David Rytz, Wouter Wolfslag, Hsiu-Chin Lin, Michael Mistry

Abstract: This paper presents a control framework to teleoperate a quadruped robot's foot for operator-guided haptic exploration of the environment. Since one leg of a quadruped robot typically only has 3 actuated degrees of freedom (DoFs), the torso is employed to assist foot posture control via a hierarchical whole-body controller. The foot and torso postures are controlled by two analytical Cartesian imp… ▽ More This paper presents a control framework to teleoperate a quadruped robot's foot for operator-guided haptic exploration of the environment. Since one leg of a quadruped robot typically only has 3 actuated degrees of freedom (DoFs), the torso is employed to assist foot posture control via a hierarchical whole-body controller. The foot and torso postures are controlled by two analytical Cartesian impedance controllers cascaded by a null space projector. The contact forces acting on supporting feet are optimized by quadratic programming (QP). The foot's Cartesian impedance controller may also estimate contact forces from trajectory tracking errors, and relay the force-feedback to the operator. A 7D haptic joystick, Sigma.7, transmits motion commands to the quadruped robot ANYmal, and renders the force feedback. Furthermore, the joystick's motion is bounded by mapping the foot's feasible force polytope constrained by the friction cones and torque limits in order to prevent the operator from driving the robot to slipping or falling over. Experimental results demonstrate the efficiency of the proposed framework. △ Less

Submitted 26 October, 2020; v1 submitted 16 December, 2019; originally announced December 2019.

Fractal Impedance for Passive Controllers: A Framework for Interaction Robotics

Authors: Keyhan Kouhkiloui Babarahmati, Carlo Tiseo, Joshua Smith, Hsiu Chin Lin, Mustafa Suphi Erden, Michael Mistry

Abstract: There is increasing interest in control frameworks capable of moving robots from industrial cages to unstructured environments and coexisting with humans. Despite significant improvement in some specific applications (e.g., medical robotics), there is still the need for a general control framework that improves interaction robustness and motion dynamics. Passive controllers show promising results… ▽ More There is increasing interest in control frameworks capable of moving robots from industrial cages to unstructured environments and coexisting with humans. Despite significant improvement in some specific applications (e.g., medical robotics), there is still the need for a general control framework that improves interaction robustness and motion dynamics. Passive controllers show promising results in this direction; however, they often rely on virtual energy tanks that can guarantee passivity as long as they do not run out of energy. In this paper, a Fractal Attractor is proposed to implement a variable impedance controller that can retain passivity without relying on energy tanks. The controller generates a Fractal Attractor around the desired state using an asymptotic stable potential field, making the controller robust to discretization and numerical integration errors. The results prove that it can accurately track both trajectories and end-effector forces during interaction. Therefore, these properties make the controller ideal for applications requiring robust dynamic interaction at the end-effector. △ Less

Submitted 27 July, 2022; v1 submitted 12 November, 2019; originally announced November 2019.

Comments: Nonlinear Dyn (2022). Video Available at https://youtu.be/Ny8zNyPS8AM

Online Simultaneous Semi-Parametric Dynamics Model Learning

Authors: Joshua Smith, Michael Mistry

Abstract: Accurate models of robots' dynamics are critical for control, stability, motion optimization, and interaction. Semi-Parametric approaches to dynamics learning combine physics-based Parametric models with unstructured Non-Parametric regression with the hope to achieve both accuracy and generalizablity. In this paper we highlight the non-stationary problem created when attempting to adapt both Param… ▽ More Accurate models of robots' dynamics are critical for control, stability, motion optimization, and interaction. Semi-Parametric approaches to dynamics learning combine physics-based Parametric models with unstructured Non-Parametric regression with the hope to achieve both accuracy and generalizablity. In this paper we highlight the non-stationary problem created when attempting to adapt both Parametric and Non-Parametric components simultaneously. We present a consistency transform designed to compensate for this non-stationary effect, such that the contributions of both models can adapt simultaneously without adversely affecting the performance of the platform. Thus we are able to apply the Semi-Parametric learning approach for continuous iterative online adaptation, without relying on batch or offline updates. We validate the transform via a perfect virtual model as well as by applying the overall system on a Kuka LWR IV manipulator. We demonstrate improved tracking performance during online learning and show a clear transference of contribution between the two components with a learning bias towards the Parametric component. △ Less

Submitted 28 January, 2020; v1 submitted 9 October, 2019; originally announced October 2019.

Comments: \c{opyright} 2020 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works

Approximation Algorithms for Process Systems Engineering

Authors: Dimitrios Letsios, Radu Baltean-Lugojan, Francesco Ceccon, Miten Mistry, Johannes Wiebe, Ruth Misener

Abstract: Designing and analyzing algorithms with provable performance guarantees enables efficient optimization problem solving in different application domains, e.g.\ communication networks, transportation, economics, and manufacturing. Despite the significant contributions of approximation algorithms in engineering, only limited and isolated works contribute from this perspective in process systems engin… ▽ More Designing and analyzing algorithms with provable performance guarantees enables efficient optimization problem solving in different application domains, e.g.\ communication networks, transportation, economics, and manufacturing. Despite the significant contributions of approximation algorithms in engineering, only limited and isolated works contribute from this perspective in process systems engineering. The current paper discusses three representative, NP-hard problems in process systems engineering: (i) pooling, (ii) process scheduling, and (iii) heat exchanger network synthesis. We survey relevant results and raise major open questions. Further, we present approximation algorithms applications which are relevant to process systems engineering: (i) better mathematical modeling, (ii) problem classification, (iii) designing solution methods, and (iv) dealing with uncertainty. This paper aims to motivate further research at the intersection of approximation algorithms and process systems engineering. △ Less

Submitted 26 September, 2019; originally announced September 2019.

Learning Generalisable Coupling Terms for Obstacle Avoidance via Low-dimensional Geometric Descriptors

Authors: Èric Pairet, Paola Ardón, Michael Mistry, Yvan Petillot

Abstract: Unforeseen events are frequent in the real-world environments where robots are expected to assist, raising the need for fast replanning of the policy in execution to guarantee the system and environment safety. Inspired by human behavioural studies of obstacle avoidance and route selection, this paper presents a hierarchical framework which generates reactive yet bounded obstacle avoidance behavio… ▽ More Unforeseen events are frequent in the real-world environments where robots are expected to assist, raising the need for fast replanning of the policy in execution to guarantee the system and environment safety. Inspired by human behavioural studies of obstacle avoidance and route selection, this paper presents a hierarchical framework which generates reactive yet bounded obstacle avoidance behaviours through a multi-layered analysis. The framework leverages the strengths of learning techniques and the versatility of dynamic movement primitives to efficiently unify perception, decision, and action levels via low-dimensional geometric descriptors of the environment. Experimental evaluation on synthetic environments and a real anthropomorphic manipulator proves that the robustness and generalisation capabilities of the proposed approach regardless of the obstacle avoidance scenario makes it suitable for robotic systems in real-world environments. △ Less

Submitted 24 June, 2019; originally announced June 2019.

Comments: Accepted for publication at the IEEE/RSJ International Conference on Intelligent Robots and Systems 2019

Learning and Composing Primitive Skills for Dual-arm Manipulation

Authors: Èric Pairet, Paola Ardón, Michael Mistry, Yvan Petillot

Abstract: In an attempt to confer robots with complex manipulation capabilities, dual-arm anthropomorphic systems have become an important research topic in the robotics community. Most approaches in the literature rely upon a great understanding of the dynamics underlying the system's behaviour and yet offer limited autonomous generalisation capabilities. To address these limitations, this work proposes a… ▽ More In an attempt to confer robots with complex manipulation capabilities, dual-arm anthropomorphic systems have become an important research topic in the robotics community. Most approaches in the literature rely upon a great understanding of the dynamics underlying the system's behaviour and yet offer limited autonomous generalisation capabilities. To address these limitations, this work proposes a modelisation for dual-arm manipulators based on dynamic movement primitives laying in two orthogonal spaces. The modularity and learning capabilities of this model are leveraged to formulate a novel end-to-end learning-based framework which (i) learns a library of primitive skills from human demonstrations, and (ii) composes such knowledge simultaneously and sequentially to confront novel scenarios. The feasibility of the proposal is evaluated by teaching the iCub humanoid the basic skills to succeed on simulated dual-arm pick-and-place tasks. The results suggest the learning and generalisation capabilities of the proposed framework extend to autonomously conduct undemonstrated dual-arm manipulation tasks. △ Less

Submitted 25 May, 2019; originally announced May 2019.

Comments: Annual Conference Towards Autonomous Robotic Systems (TAROS19)

Learning and Generalisation of Primitives Skills Towards Robust Dual-arm Manipulation

Authors: Èric Pairet, Paola Ardón, Frank Broz, Michael Mistry, Yvan Petillot

Abstract: Robots are becoming a vital ingredient in society. Some of their daily tasks require dual-arm manipulation skills in the rapidly changing, dynamic and unpredictable real-world environments where they have to operate. Given the expertise of humans in conducting these activities, it is natural to study humans' motions to use the resulting knowledge in robotic control. With this in mind, this work le… ▽ More Robots are becoming a vital ingredient in society. Some of their daily tasks require dual-arm manipulation skills in the rapidly changing, dynamic and unpredictable real-world environments where they have to operate. Given the expertise of humans in conducting these activities, it is natural to study humans' motions to use the resulting knowledge in robotic control. With this in mind, this work leverages human knowledge to formulate a more general, real-time, and less task-specific framework for dual-arm manipulation. The proposed framework is evaluated on the iCub humanoid robot and several synthetic experiments, by conducting a dual-arm pick-and-place task of a parcel in the presence of unexpected obstacles. Results suggest the suitability of the method towards robust and generalisable dual-arm manipulation. △ Less

Submitted 2 April, 2019; originally announced April 2019.

Journal ref: In Proceedings of the AAAI Fall Symposium on Reasoning and Learning in Real-World Systems for Long-Term Autonomy, 5-12. AAAI Press. 2018

Analytic Model for Quadruped Locomotion Task-Space Planning

Authors: Carlo Tiseo, Sethu Vijayakumar, Michael Mistry

Abstract: Despite the extensive presence of the legged locomotion in animals, it is extremely challenging to be reproduced with robots. Legged locomotion is an dynamic task which benefits from a planning that takes advantage of the gravitational pull on the system. However, the computational cost of such optimization rapidly increases with the complexity of kinematic structures, rendering impossible real-ti… ▽ More Despite the extensive presence of the legged locomotion in animals, it is extremely challenging to be reproduced with robots. Legged locomotion is an dynamic task which benefits from a planning that takes advantage of the gravitational pull on the system. However, the computational cost of such optimization rapidly increases with the complexity of kinematic structures, rendering impossible real-time deployment in unstructured environments. This paper proposes a simplified method that can generate desired centre of mass and feet trajectory for quadrupeds. The model describes a quadruped as two bipeds connected via their centres of mass, and it is based on the extension of an algebraic bipedal model that uses the topology of the gravitational attractor to describe bipedal locomotion strategies. The results show that the model generates trajectories that agrees with previous studies. The model will be deployed in the future as seed solution for whole-body trajectory optimization in the attempt to reduce the computational cost and obtain real-time planning of complex action in challenging environments. △ Less

Submitted 23 April, 2019; v1 submitted 19 February, 2019; originally announced February 2019.

Comments: Accepted to be Published in 2019, 41th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Berlin Germany

Journal ref: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)

Exact Lexicographic Scheduling and Approximate Rescheduling

Authors: Dimitrios Letsios, Miten Mistry, Ruth Misener

Abstract: In industrial resource allocation problems, an initial planning stage may solve a nominal problem instance and a subsequent recovery stage may intervene to repair inefficiencies and infeasibilities due to uncertainty, e.g.\ machine failures and job processing time variations. In this context, we investigate the minimum makespan scheduling problem, a.k.a.\ $P||C_{\max}$, under uncertainty. We propo… ▽ More In industrial resource allocation problems, an initial planning stage may solve a nominal problem instance and a subsequent recovery stage may intervene to repair inefficiencies and infeasibilities due to uncertainty, e.g.\ machine failures and job processing time variations. In this context, we investigate the minimum makespan scheduling problem, a.k.a.\ $P||C_{\max}$, under uncertainty. We propose a two-stage robust scheduling approach where first-stage decisions are computed with exact lexicographic scheduling and second-stage decisions are derived using approximate rescheduling. We explore recovery strategies accounting for planning decisions and constrained by limited permitted deviations from the original schedule. Our approach is substantiated analytically, with a price of robustness characterization parameterized by the degree of uncertainty, and numerically. This analysis is based on optimal substructure imposed by lexicographic optimality. Thus, lexicographic optimization enables more efficient rescheduling. Further, we revisit state-of-the-art exact lexicographic optimization methods and propose a lexicographic branch-and-bound algorithm whose performance is validated computationally. △ Less

Submitted 28 August, 2020; v1 submitted 9 May, 2018; originally announced May 2018.

Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded

Authors: Miten Mistry, Dimitrios Letsios, Gerhard Krennrich, Robert M. Lee, Ruth Misener

Abstract: Decision trees usefully represent sparse, high dimensional and noisy data. Having learned a function from this data, we may want to thereafter integrate the function into a larger decision-making problem, e.g., for picking the best chemical process catalyst. We study a large-scale, industrially-relevant mixed-integer nonlinear nonconvex optimization problem involving both gradient-boosted trees an… ▽ More Decision trees usefully represent sparse, high dimensional and noisy data. Having learned a function from this data, we may want to thereafter integrate the function into a larger decision-making problem, e.g., for picking the best chemical process catalyst. We study a large-scale, industrially-relevant mixed-integer nonlinear nonconvex optimization problem involving both gradient-boosted trees and penalty functions mitigating risk. This mixed-integer optimization problem with convex penalty terms broadly applies to optimizing pre-trained regression tree models. Decision makers may wish to optimize discrete models to repurpose legacy predictive models, or they may wish to optimize a discrete model that particularly well-represents a data set. We develop several heuristic methods to find feasible solutions, and an exact, branch-and-bound algorithm leveraging structural properties of the gradient-boosted trees and penalty functions. We computationally test our methods on concrete mixture design instance and a chemical catalysis industrial instance. △ Less

Submitted 25 September, 2019; v1 submitted 2 March, 2018; originally announced March 2018.

Uncertainty Averse Pushing with Model Predictive Path Integral Control

Authors: Ermano Arruda, Michael J Mathew, Marek Kopicki, Michael Mistry, Morteza Azad, Jeremy L Wyatt

Abstract: Planning robust robot manipulation requires good forward models that enable robust plans to be found. This work shows how to achieve this using a forward model learned from robot data to plan push manipulations. We explore learning methods (Gaussian Process Regression, and an Ensemble of Mixture Density Networks) that give estimates of the uncertainty in their predictions. These learned models are… ▽ More Planning robust robot manipulation requires good forward models that enable robust plans to be found. This work shows how to achieve this using a forward model learned from robot data to plan push manipulations. We explore learning methods (Gaussian Process Regression, and an Ensemble of Mixture Density Networks) that give estimates of the uncertainty in their predictions. These learned models are utilised by a model predictive path integral (MPPI) controller to plan how to push the box to a goal location. The planner avoids regions of high predictive uncertainty in the forward model. This includes both inherent uncertainty in dynamics, and meta uncertainty due to limited data. Thus, pushing tasks are completed in a robust fashion with respect to estimated uncertainty in the forward model and without the need of differentiable cost functions. We demonstrate the method on a real robot, and show that learning can outperform physics simulation. Using simulation, we also show the ability to plan uncertainty averse paths. △ Less

Submitted 15 October, 2017; v1 submitted 11 October, 2017; originally announced October 2017.

Comments: Humanoids 2017. Supplementary video: https://youtu.be/LjYruxwxkPM

A Projected Inverse Dynamics Approach for Dual-arm Cartesian Impedance Control

Authors: Hsiu-Chin Lin, Joshua Smith, Keyhan Kouhkiloui Babarahmati, Niels Dehio, Michael Mistry

Abstract: We propose a method for dual-arm manipulation of rigid objects, subject to external disturbance. The problem is formulated as a Cartesian impedance controller within a projected inverse dynamics framework. We use the constrained component of the controller to enforce contact and the unconstrained controller to accomplish the task with a desired 6-DOF impedance behaviour. Furthermore, the proposed… ▽ More We propose a method for dual-arm manipulation of rigid objects, subject to external disturbance. The problem is formulated as a Cartesian impedance controller within a projected inverse dynamics framework. We use the constrained component of the controller to enforce contact and the unconstrained controller to accomplish the task with a desired 6-DOF impedance behaviour. Furthermore, the proposed method optimises the torque required to maintain contact, subject to unknown disturbances, and can do so without direct measurement of external force. The techniques are evaluated on a single-arm wiping a table and a dual-arm platform manipulating a rigid object of unknown mass and with human interaction. △ Less

Submitted 3 July, 2017; originally announced July 2017.

Comments: Under review