-
And this is where we fu***d up! Lessons learned from Participatory Design in Digital Civic Initiatives
Authors:
Clara Rosa Cardoso,
Sarah Rüller,
Ana O Henriques,
Anna R L Carter,
Markus Rohde
Abstract:
Participatory design in digital civics aims to foster mutual learning and co-creation between public services and citizens. However, rarely do we collectively explore the challenges and failures we experience within PD and digital civics, to enable us to grow as a community. This workshop will explore real-world experiences that had to adapt to unforeseen circumstances. Through case presentations…
▽ More
Participatory design in digital civics aims to foster mutual learning and co-creation between public services and citizens. However, rarely do we collectively explore the challenges and failures we experience within PD and digital civics, to enable us to grow as a community. This workshop will explore real-world experiences that had to adapt to unforeseen circumstances. Through case presentations and thematic group discussions, participants will reflect on the challenges faced, the causes that led to these challenges, and collaboratively problem-solve effective solutions. Furthermore, we aim to discuss well-being impact on researchers and communities when faced with these obstacles, the strategies participants use to overcome them and how this can be fed back into the digital civics community. By that, the workshop seeks to foster dialogue, reflection, and collective learning, empowering participants with insights to navigate complexities effectively and promote resilient design practices in digital civics.
△ Less
Submitted 6 July, 2024;
originally announced July 2024.
-
Scalable End-to-End ML Platforms: from AutoML to Self-serve
Authors:
Igor L. Markov,
Pavlos A. Apostolopoulos,
Mia R. Garrard,
Tanya Qie,
Yin Huang,
Tanvi Gupta,
Anika Li,
Cesar Cardoso,
George Han,
Ryan Maghsoudian,
Norm Zhou
Abstract:
ML platforms help enable intelligent data-driven applications and maintain them with limited engineering effort. Upon sufficiently broad adoption, such platforms reach economies of scale that bring greater component reuse while improving efficiency of system development and maintenance. For an end-to-end ML platform with broad adoption, scaling relies on pervasive ML automation and system integrat…
▽ More
ML platforms help enable intelligent data-driven applications and maintain them with limited engineering effort. Upon sufficiently broad adoption, such platforms reach economies of scale that bring greater component reuse while improving efficiency of system development and maintenance. For an end-to-end ML platform with broad adoption, scaling relies on pervasive ML automation and system integration to reach the quality we term self-serve that we define with ten requirements and six optional capabilities. With this in mind, we identify long-term goals for platform development, discuss related tradeoffs and future work. Our reasoning is illustrated on two commercially-deployed end-to-end ML platforms that host hundreds of real-time use cases -- one general-purpose and one specialized.
△ Less
Submitted 3 March, 2023; v1 submitted 27 February, 2023;
originally announced February 2023.
-
Extending Attack-Fault Trees with Runtime Verification
Authors:
Rafael C. Cardoso,
Angelo Ferrando,
Michael Fisher
Abstract:
Autonomous systems are often complex and prone to software failures and cyber-attacks. We introduce RVAFTs, an extension of Attack-Fault Trees (AFTs) with runtime events that can be used to construct runtime monitors. These monitors are able to detect when failures, that can be caused either by an attack or by a fault, occur. The safety and security properties monitored are, in turn, derived from…
▽ More
Autonomous systems are often complex and prone to software failures and cyber-attacks. We introduce RVAFTs, an extension of Attack-Fault Trees (AFTs) with runtime events that can be used to construct runtime monitors. These monitors are able to detect when failures, that can be caused either by an attack or by a fault, occur. The safety and security properties monitored are, in turn, derived from the hierarchical decomposition of RVAFTs. Our approach not only provides further use of AFTs, but also improves the process of instrumentation often required in runtime verification. We explain the principles and provide a simple case study demonstrating how RVAFTs can be used in practice. Through this we are also able to evaluate the detection of faults and attacks as well as assessing the computational overhead of the monitors.
△ Less
Submitted 28 September, 2022;
originally announced September 2022.
-
A Compositional Approach to Verifying Modular Robotic Systems
Authors:
Matt Luckcuck,
Marie Farrell,
Angelo Ferrando,
Rafael C. Cardoso,
Louise A. Dennis,
Michael Fisher
Abstract:
Robotic systems used in safety-critical industrial situations often rely on modular software architectures, and increasingly include autonomous components. Verifying that these modular robotic systems behave as expected requires approaches that can cope with, and preferably take advantage of, this inherent modularity. This paper describes a compositional approach to specifying the nodes in robotic…
▽ More
Robotic systems used in safety-critical industrial situations often rely on modular software architectures, and increasingly include autonomous components. Verifying that these modular robotic systems behave as expected requires approaches that can cope with, and preferably take advantage of, this inherent modularity. This paper describes a compositional approach to specifying the nodes in robotic systems built using the Robotic Operating System (ROS), where each node is specified using First-Order Logic (FOL) assume-guarantee contracts that link the specification to the ROS implementation. We introduce inference rules that facilitate the composition of these node-level contracts to derive system-level properties. We also present a novel Domain-Specific Language, the ROS Contract Language, which captures a node's FOL specification and links this contract to its implementation. RCL contracts can be automatically translated, by our tool Vanda, into executable monitors; which we use to verify the contracts at runtime. We illustrate our approach through the specification and verification of an autonomous rover engaged in the remote inspection of a nuclear site, and finish with smaller examples that illustrate other useful features of our framework.
△ Less
Submitted 30 November, 2023; v1 submitted 10 August, 2022;
originally announced August 2022.
-
Proceedings of the Second Workshop on Agents and Robots for reliable Engineered Autonomy
Authors:
Rafael C. Cardoso,
Angelo Ferrando,
Fabio Papacchini,
Mehrnoosh Askarpour,
Louise A. Dennis
Abstract:
This volume contains the proceedings of the Second Workshop on Agents and Robots for reliable Engineered Autonomy (AREA 2022), co-located with the 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (IJCAI-ECAI 2022). The AREA workshop brings together researchers from autonomous agents, software engineering and robotic communit…
▽ More
This volume contains the proceedings of the Second Workshop on Agents and Robots for reliable Engineered Autonomy (AREA 2022), co-located with the 31st International Joint Conference on Artificial Intelligence and the 25th European Conference on Artificial Intelligence (IJCAI-ECAI 2022). The AREA workshop brings together researchers from autonomous agents, software engineering and robotic communities, as combining knowledge coming from these research areas may lead to innovative approaches that solve complex problems related with the verification and validation of autonomous robotic systems.
△ Less
Submitted 19 July, 2022;
originally announced July 2022.
-
Towards Partial Monitoring: It is Always too Soon to Give Up
Authors:
Angelo Ferrando,
Rafael C. Cardoso
Abstract:
Runtime Verification is a lightweight formal verification technique. It is used to verify at runtime whether the system under analysis behaves as expected. The expected behaviour is usually formally specified by means of properties, which are used to automatically synthesise monitors. A monitor is a device that, given a sequence of events representing a system execution, returns a verdict symbolis…
▽ More
Runtime Verification is a lightweight formal verification technique. It is used to verify at runtime whether the system under analysis behaves as expected. The expected behaviour is usually formally specified by means of properties, which are used to automatically synthesise monitors. A monitor is a device that, given a sequence of events representing a system execution, returns a verdict symbolising the satisfaction or violation of the formal property. Properties that can (resp. cannot) be verified at runtime by a monitor are called monitorable and non-monitorable, respectively. In this paper, we revise the notion of monitorability from a practical perspective, where we show how non-monitorable properties can still be used to generate partial monitors, which can partially check the properties. Finally, we present the implications both from a theoretical and practical perspectives.
△ Less
Submitted 24 October, 2021;
originally announced October 2021.
-
MLFC: From 10 to 50 Planners in the Multi-Agent Programming Contest
Authors:
Rafael C. Cardoso,
Angelo Ferrando,
Fabio Papacchini,
Matt Luckcuck,
Sven Linker,
Terry R. Payne
Abstract:
In this paper, we describe the strategies used by our team, MLFC, that led us to achieve the 2nd place in the 15th edition of the Multi-Agent Programming Contest. The scenario used in the contest is an extension of the previous edition (14th) "Agents Assemble" wherein two teams of agents move around a 2D grid and compete to assemble complex block structures. We discuss the languages and tools used…
▽ More
In this paper, we describe the strategies used by our team, MLFC, that led us to achieve the 2nd place in the 15th edition of the Multi-Agent Programming Contest. The scenario used in the contest is an extension of the previous edition (14th) "Agents Assemble" wherein two teams of agents move around a 2D grid and compete to assemble complex block structures. We discuss the languages and tools used during the development of our team. Then, we summarise the main strategies that were carried over from our previous participation in the 14th edition and list the limitations (if any) of using these strategies in the latest contest edition. We also developed new strategies that were made specifically for the extended scenario: cartography (determining the size of the map); formal verification of the map merging protocol (to provide assurances that it works when increasing the number of agents); plan cache (efficiently scaling the number of planners); task achievement (forming groups of agents to achieve tasks); and bullies (agents that focus on stopping agents from the opposing team). Finally, we give a brief overview of our performance in the contest and discuss what we believe were our shortcomings.
△ Less
Submitted 18 October, 2021; v1 submitted 15 October, 2021;
originally announced October 2021.
-
Formal Verification of a Map Merging Protocol in the Multi-Agent Programming Contest
Authors:
Matt Luckcuck,
Rafael C. Cardoso
Abstract:
Communication is a critical part of enabling multi-agent systems to cooperate. This means that applying formal methods to protocols governing communication within multi-agent systems provides useful confidence in its reliability. In this paper, we describe the formal verification of a complex communication protocol that coordinates agents merging maps of their environment. The protocol was used by…
▽ More
Communication is a critical part of enabling multi-agent systems to cooperate. This means that applying formal methods to protocols governing communication within multi-agent systems provides useful confidence in its reliability. In this paper, we describe the formal verification of a complex communication protocol that coordinates agents merging maps of their environment. The protocol was used by the LFC team in the 2019 edition of the Multi-Agent Programming Contest (MAPC). Our specification of the protocol is written in Communicating Sequential Processes (CSP), which is a well-suited approach to specifying agent communication protocols due to its focus on concurrent communicating systems. We validate the specification's behaviour using scenarios where the correct behaviour is known, and verify that eventually all the maps have merged.
△ Less
Submitted 22 June, 2021; v1 submitted 8 June, 2021;
originally announced June 2021.
-
Towards Compositional Verification for Modular Robotic Systems
Authors:
Rafael C. Cardoso,
Louise A. Dennis,
Marie Farrell,
Michael Fisher,
Matt Luckcuck
Abstract:
Software engineering of modular robotic systems is a challenging task, however, verifying that the developed components all behave as they should individually and as a whole presents its own unique set of challenges. In particular, distinct components in a modular robotic system often require different verification techniques to ensure that they behave as expected. Ensuring whole system consistenc…
▽ More
Software engineering of modular robotic systems is a challenging task, however, verifying that the developed components all behave as they should individually and as a whole presents its own unique set of challenges. In particular, distinct components in a modular robotic system often require different verification techniques to ensure that they behave as expected. Ensuring whole system consistency when individual components are verified using a variety of techniques and formalisms is difficult. This paper discusses how to use compositional verification to integrate the various verification techniques that are applied to modular robotic software, using a First-Order Logic (FOL) contract that captures each component's assumptions and guarantees. These contracts can then be used to guide the verification of the individual components, be it by testing or the use of a formal method. We provide an illustrative example of an autonomous robot used in remote inspection. We also discuss a way of defining confidence for the verification associated with each component.
△ Less
Submitted 2 December, 2020;
originally announced December 2020.
-
Adaptable and Verifiable BDI Reasoning
Authors:
Peter Stringer,
Rafael C. Cardoso,
Xiaowei Huang,
Louise A. Dennis
Abstract:
Long-term autonomy requires autonomous systems to adapt as their capabilities no longer perform as expected. To achieve this, a system must first be capable of detecting such changes. In this position paper, we describe a system architecture for BDI autonomous agents capable of adapting to changes in a dynamic environment and outline the required research. Specifically, we describe an agent-mainta…
▽ More
Long-term autonomy requires autonomous systems to adapt as their capabilities no longer perform as expected. To achieve this, a system must first be capable of detecting such changes. In this position paper, we describe a system architecture for BDI autonomous agents capable of adapting to changes in a dynamic environment and outline the required research. Specifically, we describe an agent-maintained self-model with accompanying theories of durative actions and learning new action descriptions in BDI systems.
△ Less
Submitted 22 July, 2020;
originally announced July 2020.
-
Proceedings of the First Workshop on Agents and Robots for reliable Engineered Autonomy
Authors:
Rafael C. Cardoso,
Angelo Ferrando,
Daniela Briola,
Claudio Menghi,
Tobias Ahlbrecht
Abstract:
This volume contains the proceedings of the First Workshop on Agents and Robots for reliable Engineered Autonomy (AREA 2020), co-located with the 24th European Conference on Artificial Intelligence (ECAI 2020). AREA brings together researchers from autonomous agents, software engineering and robotic communities, as combining knowledge coming from these research areas may lead to innovative approac…
▽ More
This volume contains the proceedings of the First Workshop on Agents and Robots for reliable Engineered Autonomy (AREA 2020), co-located with the 24th European Conference on Artificial Intelligence (ECAI 2020). AREA brings together researchers from autonomous agents, software engineering and robotic communities, as combining knowledge coming from these research areas may lead to innovative approaches that solve complex problems related with the verification and validation of autonomous robotic systems.
△ Less
Submitted 22 July, 2020;
originally announced July 2020.
-
Heterogeneous Verification of an Autonomous Curiosity Rover
Authors:
Rafael C. Cardoso,
Marie Farrell,
Matt Luckcuck,
Angelo Ferrando,
Michael Fisher
Abstract:
The Curiosity rover is one of the most complex systems successfully deployed in a planetary exploration mission to date. It was sent by NASA to explore the surface of Mars and to identify potential signs of life. Even though it has limited autonomy on-board, most of its decisions are made by the ground control team. This hinders the speed at which the Curiosity reacts to its environment, due to th…
▽ More
The Curiosity rover is one of the most complex systems successfully deployed in a planetary exploration mission to date. It was sent by NASA to explore the surface of Mars and to identify potential signs of life. Even though it has limited autonomy on-board, most of its decisions are made by the ground control team. This hinders the speed at which the Curiosity reacts to its environment, due to the communication delays between Earth and Mars. Depending on the orbital position of both planets, it can take 4--24 minutes for a message to be transmitted between Earth and Mars. If the Curiosity were controlled autonomously, it would be able to perform its activities much faster and more flexibly. However, one of the major barriers to increased use of autonomy in such scenarios is the lack of assurances that the autonomous behaviour will work as expected. In this paper, we use a Robot Operating System (ROS) model of the Curiosity that is simulated in Gazebo and add an autonomous agent that is responsible for high-level decision-making. Then, we use a mixture of formal and non-formal techniques to verify the distinct system components (ROS nodes). This use of heterogeneous verification techniques is essential to provide guarantees about the nodes at different abstraction levels, and allows us to bring together relevant verification evidence to provide overall assurance.
△ Less
Submitted 20 July, 2020;
originally announced July 2020.
-
LFC: Combining Autonomous Agents and Automated Planning in the Multi-Agent Programming Contest
Authors:
Rafael C. Cardoso,
Angelo Ferrando,
Fabio Papacchini
Abstract:
The 2019 Multi-Agent Programming Contest introduced a new scenario, Agents Assemble, where two teams of agents move around a 2D grid and compete to assemble complex block structures. In this paper, we describe the strategies used by our team that led us to achieve first place in the contest. Our strategies tackle some of the major challenges in the 2019 contest: how to explore and build a map when…
▽ More
The 2019 Multi-Agent Programming Contest introduced a new scenario, Agents Assemble, where two teams of agents move around a 2D grid and compete to assemble complex block structures. In this paper, we describe the strategies used by our team that led us to achieve first place in the contest. Our strategies tackle some of the major challenges in the 2019 contest: how to explore and build a map when agents only have access to local vision and no global coordinates; how to move around the map efficiently even though there are dynamic events that can change the cells in the grid; and how to assemble and submit complex block structures given that the opposing team may try to sabotage us. To implement our strategies, we use the multi-agent systems development platform JaCaMo to program our agents and the Fast Downward planner to plan the movement of the agent in the grid. We also provide a brief discussion of our matches in the contest and give our analysis of how our team performed in each match.
△ Less
Submitted 4 June, 2020;
originally announced June 2020.
-
Citation network centrality: a scientific awards predictor?
Authors:
Osame Kinouchi,
Adriano J. Holanda,
George C. Cardoso
Abstract:
The $K$-index is an easily computable centrality index in complex networks, such as a scientific citations network. A researcher has a $K$-index equal to $K$ if he or she is cited by $K$ articles that have at least $K$ citations. The $K$-index has several advantages over Hirsh's $h$-index and, in previous studies, has shown better correlation with Nobel prizes than any other index given by the {\e…
▽ More
The $K$-index is an easily computable centrality index in complex networks, such as a scientific citations network. A researcher has a $K$-index equal to $K$ if he or she is cited by $K$ articles that have at least $K$ citations. The $K$-index has several advantages over Hirsh's $h$-index and, in previous studies, has shown better correlation with Nobel prizes than any other index given by the {\em Web of Science}, including the $h$-index. It is plausible that researchers who are the most connected to other scientifically well-connected researchers are the most likely to be doing important work and more likely to be awarded major prizes in a given area. However, the correlation found does not imply causation. Here we perform an experiment using the $K$-index, producing a shortlist of twelve candidates for major scientific prizes, including the Physics Nobel award, in the near future. For example, our top-12 $K$-index list naturally selects the 2019 Nobel laureate, James Peebles. The list can be updated annually and should be compared to laureates of the following years
△ Less
Submitted 18 October, 2019; v1 submitted 6 October, 2019;
originally announced October 2019.
-
Modular Verification of Autonomous Space Robotics
Authors:
Marie Farrell,
Rafael C. Cardoso,
Louise A. Dennis,
Clare Dixon,
Michael Fisher,
Georgios Kourtis,
Alexei Lisitsa,
Matt Luckcuck,
Matt Webster
Abstract:
Ensuring that autonomous space robot control software behaves as it should is crucial, particularly as software failure in space often equates to mission failure and could potentially endanger nearby astronauts and costly equipment. To minimise mission failure caused by software errors, we can utilise a variety of tools and techniques to verify that the software behaves as intended. In particular,…
▽ More
Ensuring that autonomous space robot control software behaves as it should is crucial, particularly as software failure in space often equates to mission failure and could potentially endanger nearby astronauts and costly equipment. To minimise mission failure caused by software errors, we can utilise a variety of tools and techniques to verify that the software behaves as intended. In particular, distinct nodes in a robotic system often require different verification techniques to ensure that they behave as expected. This paper introduces a method for integrating the various verification techniques that are applied to robotic software, via a First-Order Logic (FOL) specification that captures each node's assumptions and guarantees. These FOL specifications are then used to guide the verification of the individual nodes, be it by testing or the use of a formal method. We also outline a way of measuring our confidence in the verification of the entire system in terms of the verification techniques used.
△ Less
Submitted 28 August, 2019;
originally announced August 2019.
-
A simple centrality index for scientific social recognition
Authors:
Osame Kinouchi,
Leonardo D. H. Soares,
George C. Cardoso
Abstract:
We introduce a new centrality index for bipartite network of papers and authors that we call $K$-index. The $K$-index grows with the citation performance of the papers that cite a given researcher and can seen as a measure of scientific social recognition. Indeed, the $K$-index measures the number of hubs, defined in a self-consistent way in the bipartite network, that cites a given author. We sho…
▽ More
We introduce a new centrality index for bipartite network of papers and authors that we call $K$-index. The $K$-index grows with the citation performance of the papers that cite a given researcher and can seen as a measure of scientific social recognition. Indeed, the $K$-index measures the number of hubs, defined in a self-consistent way in the bipartite network, that cites a given author. We show that the $K$-index can be computed by simple inspection of the Web of Science platform and presents several advantages over other centrality indexes, in particular Hirsch $h$-index. The $K$-index is robust to self-citations, is not limited by the total number of papers published by a researcher as occurs for the $h$-index and can distinguish in a consistent way researchers that have the same $h$-index but very different scientific social recognition. The $K$-index easily detects a known case of a researcher with inflated number of papers, citations and $h$-index due to scientific misconduct. Finally, we show that, in a sample of twenty-eight physics Nobel laureates and twenty-eight highly cited non-Nobel-laureate physicists, the $K$-index correlates better to the achievement of the prize than the number of papers, citations, citations per paper, citing articles or the $h$-index. Clustering researchers in a $K$ versus $h$ plot reveals interesting outliers that suggest that these two indexes can present complementary independent information.
△ Less
Submitted 28 September, 2017; v1 submitted 16 September, 2016;
originally announced September 2016.
-
Uplink Performance Evaluation of Massive MU-MIMO Systems
Authors:
Felipe A. P. de Figueiredo,
Joao Paulo Miranda,
Fabricio L. Figueiredo,
Fabbryccio A. C. M. Cardoso
Abstract:
The present paper deals with an OFDM-based uplink within a multi-user MIMO (MU-MIMO) system where a massive MIMO approach is employed. In this context, the linear detectors Minimum Mean-Squared Error (MMSE), Zero Forcing (ZF) and Maximum Ratio Combining (MRC) are considered and assessed. This papers includes Bit Error Rate (BER) results for uncoded QPSK/OFDM transmissions through a flat Rayleigh f…
▽ More
The present paper deals with an OFDM-based uplink within a multi-user MIMO (MU-MIMO) system where a massive MIMO approach is employed. In this context, the linear detectors Minimum Mean-Squared Error (MMSE), Zero Forcing (ZF) and Maximum Ratio Combining (MRC) are considered and assessed. This papers includes Bit Error Rate (BER) results for uncoded QPSK/OFDM transmissions through a flat Rayleigh fading channel under the assumption of perfect power control and channel estimation. BER results are obtained through Monte Carlo simulations. Performance results are discussed in detail and we confirm the achievable "massive MIMO" effects, even for a reduced complexity detection technique, when the number of receive antennas at BS is much larger than the number of transmit antennas.
△ Less
Submitted 7 March, 2015;
originally announced March 2015.