Showing 1–50 of 202 results for author: Halpern, J

A Knowledge-Based Analysis of Intersection Protocols

Authors: Kaya Alpturer, Joseph Y. Halpern, Ron van der Meyden

Abstract: The increasing wireless communication capabilities of vehicles creates opportunities for more efficient intersection management strategies. One promising approach is the replacement of traffic lights with a system wherein vehicles run protocols among themselves to determine right of way. In this paper, we define the intersection problem to model this scenario abstractly, without any assumptions on… ▽ More The increasing wireless communication capabilities of vehicles creates opportunities for more efficient intersection management strategies. One promising approach is the replacement of traffic lights with a system wherein vehicles run protocols among themselves to determine right of way. In this paper, we define the intersection problem to model this scenario abstractly, without any assumptions on the specific structure of the intersection or a bound on the number of vehicles. Protocols solving the intersection problem must guarantee safety (no collisions) and liveness (every vehicle eventually goes through). In addition, we would like these protocols to satisfy various optimality criteria, some of which turn out to be achievable only in a subset of the contexts. In particular, we show a partial equivalence between eliminating unnecessary waiting, a criterion of interest in the distributed mutual-exclusion literature, and a notion of optimality that we define called lexicographical optimality. We then introduce a framework to design protocols for the intersection problem by converting an intersection policy, which is based on a global view of the intersection, to a protocol that can be run by the vehicles through the use of knowledge-based programs. Our protocols are shown to guarantee safety and liveness while also being optimal under sufficient conditions on the context. Finally, we investigate protocols in the presence of faulty vehicles that experience communication failures and older vehicles with limited communication capabilities. We show that intersection protocols can be made safe, live and optimal even in the presence of faulty behavior. △ Less

Submitted 18 August, 2024; originally announced August 2024.

Comments: The full version of the DISC 2024 paper

Paying to Do Better: Games with Payments between Learning Agents

Authors: Yoav Kolumbus, Joe Halpern, Éva Tardos

Abstract: In repeated games, such as auctions, players typically use learning algorithms to choose their actions. The use of such autonomous learning agents has become widespread on online platforms. In this paper, we explore the impact of players incorporating monetary transfers into their agents' algorithms, aiming to incentivize behavior in their favor. Our focus is on understanding when players have inc… ▽ More In repeated games, such as auctions, players typically use learning algorithms to choose their actions. The use of such autonomous learning agents has become widespread on online platforms. In this paper, we explore the impact of players incorporating monetary transfers into their agents' algorithms, aiming to incentivize behavior in their favor. Our focus is on understanding when players have incentives to make use of monetary transfers, how these payments affect learning dynamics, and what the implications are for welfare and its distribution among the players. We propose a simple game-theoretic model to capture such scenarios. Our results on general games show that in a broad class of games, players benefit from letting their learning agents make payments to other learners during the game dynamics, and that in many cases, this kind of behavior improves welfare for all players. Our results on first- and second-price auctions show that in equilibria of the ``payment policy game,'' the agents' dynamics can reach strong collusive outcomes with low revenue for the auctioneer. These results highlight a challenge for mechanism design in systems where automated learning agents can benefit from interacting with their peers outside the boundaries of the mechanism. △ Less

Submitted 31 May, 2024; originally announced May 2024.

MSC Class: 91A05; 91A06; 91A10; 91A20; 91A40; 91A80; 91B26 ACM Class: F.0; I.2; I.2.6; J.4

Intervention and Conditioning in Causal Bayesian Networks

Authors: Sainyam Galhotra, Joseph Y. Halpern

Abstract: Causal models are crucial for understanding complex systems and identifying causal relationships among variables. Even though causal models are extremely popular, conditional probability calculation of formulas involving interventions pose significant challenges. In case of Causal Bayesian Networks (CBNs), Pearl assumes autonomy of mechanisms that determine interventions to calculate a range of pr… ▽ More Causal models are crucial for understanding complex systems and identifying causal relationships among variables. Even though causal models are extremely popular, conditional probability calculation of formulas involving interventions pose significant challenges. In case of Causal Bayesian Networks (CBNs), Pearl assumes autonomy of mechanisms that determine interventions to calculate a range of probabilities. We show that by making simple yet often realistic independence assumptions, it is possible to uniquely estimate the probability of an interventional formula (including the well-studied notions of probability of sufficiency and necessity). We discuss when these assumptions are appropriate. Importantly, in many cases of interest, when the assumptions are appropriate, these probability estimates can be evaluated using observational data, which carries immense significance in scenarios where conducting experiments is impractical or unfeasible. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems

Authors: David "davidad" Dalrymple, Joar Skalse, Yoshua Bengio, Stuart Russell, Max Tegmark, Sanjit Seshia, Steve Omohundro, Christian Szegedy, Ben Goldhaber, Nora Ammann, Alessandro Abate, Joe Halpern, Clark Barrett, Ding Zhao, Tan Zhi-Xuan, Jeannette Wing, Joshua Tenenbaum

Abstract: Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts. In this paper, we will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these appro… ▽ More Ensuring that AI systems reliably and robustly avoid harmful or dangerous behaviours is a crucial challenge, especially for AI systems with a high degree of autonomy and general intelligence, or systems used in safety-critical contexts. In this paper, we will introduce and define a family of approaches to AI safety, which we will refer to as guaranteed safe (GS) AI. The core feature of these approaches is that they aim to produce AI systems which are equipped with high-assurance quantitative safety guarantees. This is achieved by the interplay of three core components: a world model (which provides a mathematical description of how the AI system affects the outside world), a safety specification (which is a mathematical description of what effects are acceptable), and a verifier (which provides an auditable proof certificate that the AI satisfies the safety specification relative to the world model). We outline a number of approaches for creating each of these three core components, describe the main technical challenges, and suggest a number of potential solutions to them. We also argue for the necessity of this approach to AI safety, and for the inadequacy of the main alternative approaches. △ Less

Submitted 8 July, 2024; v1 submitted 10 May, 2024; originally announced May 2024.

Mathematical Explanations

Authors: Joseph Y. Halpern

Abstract: A definition of what counts as an explanation of mathematical statement, and when one explanation is better than another, is given. Since all mathematical facts must be true in all causal models, and hence known by an agent, mathematical facts cannot be part of an explanation (under the standard notion of explanation). This problem is solved using impossible possible worlds. A definition of what counts as an explanation of mathematical statement, and when one explanation is better than another, is given. Since all mathematical facts must be true in all causal models, and hence known by an agent, mathematical facts cannot be part of an explanation (under the standard notion of explanation). This problem is solved using impossible possible worlds. △ Less

Submitted 31 December, 2023; originally announced February 2024.

Explaining Image Classifiers

Authors: Hana Chockler, Joseph Y. Halpern

Abstract: We focus on explaining image classifiers, taking the work of Mothilal et al. [2021] (MMTS) as our point of departure. We observe that, although MMTS claim to be using the definition of explanation proposed by Halpern [2016], they do not quite do so. Roughly speaking, Halpern's definition has a necessity clause and a sufficiency clause. MMTS replace the necessity clause by a requirement that, as we… ▽ More We focus on explaining image classifiers, taking the work of Mothilal et al. [2021] (MMTS) as our point of departure. We observe that, although MMTS claim to be using the definition of explanation proposed by Halpern [2016], they do not quite do so. Roughly speaking, Halpern's definition has a necessity clause and a sufficiency clause. MMTS replace the necessity clause by a requirement that, as we show, implies it. Halpern's definition also allows agents to restrict the set of options considered. While these difference may seem minor, as we show, they can have a nontrivial impact on explanations. We also show that, essentially without change, Halpern's definition can handle two issues that have proved difficult for other approaches: explanations of absence (when, for example, an image classifier for tumors outputs "no tumor") and explanations of rare events (such as tumors). △ Less

Submitted 24 January, 2024; originally announced January 2024.

Subjective Causality

Authors: Joseph Y. Halpern, Evan Piermont

Abstract: We show that it is possible to understand and identify a decision maker's subjective causal judgements by observing her preferences over interventions. Following Pearl [2000], we represent causality using causal models (also called structural equations models), where the world is described by a collection of variables, related by equations. We show that if a preference relation over interventions… ▽ More We show that it is possible to understand and identify a decision maker's subjective causal judgements by observing her preferences over interventions. Following Pearl [2000], we represent causality using causal models (also called structural equations models), where the world is described by a collection of variables, related by equations. We show that if a preference relation over interventions satisfies certain axioms (related to standard axioms regarding counterfactuals), then we can define (i) a causal model, (ii) a probability capturing the decision-maker's uncertainty regarding the external factors in the world and (iii) a utility on outcomes such that each intervention is associated with an expected utility and such that intervention $A$ is preferred to $B$ iff the expected utility of $A$ is greater than that of $B$. In addition, we characterize when the causal model is unique. Thus, our results allow a modeler to test the hypothesis that a decision maker's preferences are consistent with some causal model and to identify causal judgements from observed behavior. △ Less

Submitted 17 January, 2024; originally announced January 2024.

Bounding the Communication Complexity of Fault-Tolerant Common Coin Tossing

Authors: Ivan Geffner, Joseph Y. Halpern

Abstract: Protocols for tossing a common coin play a key role in the vast majority of implementations of consensus. Even though the common coins in the literature are usually \emph{fair} (they have equal chance of landing heads or tails), we focus on the problem of implementing a \emph{biased} common coin such that the probability of landing heads is $p \in [0,1]$. Even though biased common coins can be imp… ▽ More Protocols for tossing a common coin play a key role in the vast majority of implementations of consensus. Even though the common coins in the literature are usually \emph{fair} (they have equal chance of landing heads or tails), we focus on the problem of implementing a \emph{biased} common coin such that the probability of landing heads is $p \in [0,1]$. Even though biased common coins can be implemented using fair common coins, we show that this can require significant inter-party communication. In fact, we show that there is no bound on the number of messages needed to generate a common coin of bias $p$ in a way that tolerates even one malicious agent, even if we restrict $p$ to an arbitrary infinite subset of $[0,1]$ (e.g., rational numbers of the form $1/2^n$) and assume that the system is synchronous. By way of contrast, if we do not require the protocol to tolerate a faulty agent, we can do this. Thus, the cause of the message complexity is the requirement of fault tolerance. △ Less

Submitted 24 December, 2023; v1 submitted 22 December, 2023; originally announced December 2023.

Inference for Probabilistic Dependency Graphs

Authors: Oliver E. Richardson, Joseph Y. Halpern, Christopher De Sa

Abstract: Probabilistic dependency graphs (PDGs) are a flexible class of probabilistic graphical models, subsuming Bayesian Networks and Factor Graphs. They can also capture inconsistent beliefs, and provide a way of measuring the degree of this inconsistency. We present the first tractable inference algorithm for PDGs with discrete variables, making the asymptotic complexity of PDG inference similar that o… ▽ More Probabilistic dependency graphs (PDGs) are a flexible class of probabilistic graphical models, subsuming Bayesian Networks and Factor Graphs. They can also capture inconsistent beliefs, and provide a way of measuring the degree of this inconsistency. We present the first tractable inference algorithm for PDGs with discrete variables, making the asymptotic complexity of PDG inference similar that of the graphical models they generalize. The key components are: (1) the observation that, in many cases, the distribution a PDG specifies can be formulated as a convex optimization problem (with exponential cone constraints), (2) a construction that allows us to express these problems compactly for PDGs of boundeed treewidth, (3) contributions to the theory of PDGs that justify the construction, and (4) an appeal to interior point methods that can solve such problems in polynomial time. We verify the correctness and complexity of our approach, and provide an implementation of it. We then evaluate our implementation, and demonstrate that it outperforms baseline approaches. Our code is available at http://github.com/orichardson/pdg-infer-uai. △ Less

Submitted 9 November, 2023; originally announced November 2023.

Comments: extended version of the paper with corrected reduction proof

Journal ref: PMLR 216:1741-1751, 2023

Communication games, sequential equilibrium, and mediators

Authors: Ivan Geffner, Joseph Y. Halpern

Abstract: We consider $k$-resilient sequential equilibria, strategy profiles where no player in a coalition of at most $k$ players believes that it can increase its utility by deviating, regardless of its local state. We prove that all $k$-resilient sequential equilibria that can be implemented with a trusted mediator can also be implemented without the mediator in a synchronous system of $n$ players if… ▽ More We consider $k$-resilient sequential equilibria, strategy profiles where no player in a coalition of at most $k$ players believes that it can increase its utility by deviating, regardless of its local state. We prove that all $k$-resilient sequential equilibria that can be implemented with a trusted mediator can also be implemented without the mediator in a synchronous system of $n$ players if $n >3k$. In asynchronous systems, where there is no global notion of time and messages may take arbitrarily long to get to their recipient, we prove that a $k$-resilient sequential equilibrium with a mediator can be implemented without the mediator if $n > 4k$. These results match the lower bounds given by Abraham, Dolev, and Halpern (2008) and Geffner and Halpern (2023) for implementing a Nash equilibrium without a mediator (which are easily seen to apply to implementing a sequential equilibrium) and improve the results of Gerardi, who showed that, in the case that $k=1$, a sequential equilibrium can be implemented in synchronous systems if $n \ge 5$. △ Less

Submitted 9 January, 2024; v1 submitted 25 September, 2023; originally announced September 2023.

MSC Class: 91A28 (Primary) 91A18; 91A27 (Secondary)

Chunking Tasks for Present-Biased Agents

Authors: Joe Halpern, Aditya Saraf

Abstract: Everyone puts things off sometimes. How can we combat this tendency to procrastinate? A well-known technique used by instructors is to break up a large project into more manageable chunks. But how should this be done best? Here we study the process of chunking using the graph-theoretic model of present bias introduced by Kleinberg and Oren (2014). We first analyze how to optimally chunk single edg… ▽ More Everyone puts things off sometimes. How can we combat this tendency to procrastinate? A well-known technique used by instructors is to break up a large project into more manageable chunks. But how should this be done best? Here we study the process of chunking using the graph-theoretic model of present bias introduced by Kleinberg and Oren (2014). We first analyze how to optimally chunk single edges within a task graph, given a limited number of chunks. We show that for edges on the shortest path, the optimal chunking makes initial chunks easy and later chunks progressively harder. For edges not on the shortest path, optimal chunking is significantly more complex, but we provide an efficient algorithm that chunks the edge optimally. We then use our optimal edge-chunking algorithm to optimally chunk task graphs. We show that with a linear number of chunks on each edge, the biased agent's cost can be exponentially lowered, to within a constant factor of the true cheapest path. Finally, we extend our model to the case where a task designer must chunk a graph for multiple types of agents simultaneously. The problem grows significantly more complex with even two types of agents, but we provide optimal graph chunking algorithms for two types. Our work highlights the efficacy of chunking as a means to combat present bias. △ Less

Submitted 22 September, 2023; originally announced September 2023.

Comments: Published in Economics and Computation 2023

Colordag: An Incentive-Compatible Blockchain

Authors: Ittai Abraham, Danny Dolev, Ittay Eyal, Joseph Y. Halpern

Abstract: We present Colordag, a blockchain protocol where following the prescribed strategy is, with high probability, a best response as long as all miners have less than 1/2 of the mining power. We prove the correctness of Colordag even if there is an extremely powerful adversary who knows future actions of the scheduler: specifically, when agents will generate blocks and when messages will arrive. The s… ▽ More We present Colordag, a blockchain protocol where following the prescribed strategy is, with high probability, a best response as long as all miners have less than 1/2 of the mining power. We prove the correctness of Colordag even if there is an extremely powerful adversary who knows future actions of the scheduler: specifically, when agents will generate blocks and when messages will arrive. The state-of-the-art protocol, Fruitchain, is an epsilon-Nash equilibrium as long as all miners have less than 1/2 of the mining power. However, there is a simple deviation that guarantees that deviators are never worse off than they would be by following Fruitchain, and can sometimes do better. Thus, agents are motivated to deviate. Colordag implements a solution concept that we call epsilon-sure Nash equilibrium and does not suffer from this problem. Because it is an epsilon-sure Nash equilibrium, Colordag is an epsilon Nash equilibrium and with probability (1 - epsilon) is a best response. △ Less

Submitted 22 August, 2023; originally announced August 2023.

Comments: To be published in DISC 2023

Strategic Play By Resource-Bounded Agents in Security Games

Authors: Xinming Liu, Joseph Y. Halpern

Abstract: Many studies have shown that humans are "predictably irrational": they do not act in a fully rational way, but their deviations from rational behavior are quite systematic. Our goal is to see the extent to which we can explain and justify these deviations as the outcome of rational but resource-bounded agents doing as well as they can, given their limitations. We focus on the well-studied ranger-p… ▽ More Many studies have shown that humans are "predictably irrational": they do not act in a fully rational way, but their deviations from rational behavior are quite systematic. Our goal is to see the extent to which we can explain and justify these deviations as the outcome of rational but resource-bounded agents doing as well as they can, given their limitations. We focus on the well-studied ranger-poacher game, where rangers are trying to protect a number of sites from poaching. We capture the computational limitations by modeling the poacher and the ranger as probabilistic finite automata (PFAs). We show that, with sufficiently large memory, PFAs learn to play the Nash equilibrium (NE) strategies of the game and achieve the NE utility. However, if we restrict the memory, we get more "human-like" behaviors, such as probability matching (i.e., visiting sites in proportion to the probability of a rhino being there), and avoiding sites where there was a bad outcome (e.g., the poacher was caught by the ranger), that we also observed in experiments conducted on Amazon Mechanical Turk. Interestingly, we find that adding human-like behaviors such as probability matching and overweighting significant events (like getting caught) actually improves performance, showing that this seemingly irrational behavior can be quite rational. △ Less

Submitted 25 July, 2023; originally announced July 2023.

Comments: 9 pages, 6 figures

Sequential Language-based Decisions

Authors: Adam Bjorndahl, Joseph Y. Halpern

Abstract: In earlier work, we introduced the framework of language-based decisions, the core idea of which was to modify Savage's classical decision-theoretic framework by taking actions to be descriptions in some language, rather than functions from states to outcomes, as they are defined classically. Actions had the form "if psi then do(phi)", where psi and phi were formulas in some underlying language,… ▽ More In earlier work, we introduced the framework of language-based decisions, the core idea of which was to modify Savage's classical decision-theoretic framework by taking actions to be descriptions in some language, rather than functions from states to outcomes, as they are defined classically. Actions had the form "if psi then do(phi)", where psi and phi were formulas in some underlying language, specifying what effects would be brought about under what circumstances. The earlier work allowed only one-step actions. But, in practice, plans are typically composed of a sequence of steps. Here, we extend the earlier framework to sequential actions, making it much more broadly applicable. Our technical contribution is a representation theorem in the classical spirit: agents whose preferences over actions satisfy certain constraints can be modeled as if they are expected utility maximizers. As in the earlier work, due to the language-based specification of the actions, the representation theorem requires a construction not only of the probability and utility functions representing the agent's beliefs and preferences, but also the state and outcomes spaces over which these are defined, as well as a "selection function" which intuitively captures how agents disambiguate coarse descriptions. The (unbounded) depth of action sequencing adds substantial interest (and complexity!) to the proof. △ Less

Submitted 11 July, 2023; originally announced July 2023.

Comments: In Proceedings TARK 2023, arXiv:2307.04005

Journal ref: EPTCS 379, 2023, pp. 131-141

Optimal Eventual Byzantine Agreement Protocols with Omission Failures

Authors: Kaya Alpturer, Joseph Y. Halpern, Ron van der Meyden

Abstract: Work on \emph{optimal} protocols for \emph{Eventual Byzantine Agreement} (EBA) -- protocols that, in a precise sense, decide as soon as possible in every run and guarantee that all nonfaulty agents decide on the same value -- has focused on emph{full-information protocols} (FIPs), where agents repeatedly send messages that completely describe their past observations to every other agent. While it… ▽ More Work on \emph{optimal} protocols for \emph{Eventual Byzantine Agreement} (EBA) -- protocols that, in a precise sense, decide as soon as possible in every run and guarantee that all nonfaulty agents decide on the same value -- has focused on emph{full-information protocols} (FIPs), where agents repeatedly send messages that completely describe their past observations to every other agent. While it can be shown that, without loss of generality, we can take an optimal protocol to be an FIP, full information exchange is impractical to implement for many applications due to the required message size. We separate protocols into two parts, the \emph{information-exchange protocol} and the \emph{action protocol}, so as to be able to examine the effects of more limited information exchange. We then define a notion of optimality with respect to an information-exchange protocol. Roughly speaking, an action protocol $P$ is optimal with respect to an information-exchange protocol $\mathcal{E}$ if, with $P$, agents decide as soon as possible among action protocols that exchange information according to $\mathcal{E}$. We present a knowledge-based EBA program for omission failures all of whose implementations are guaranteed to be correct and are optimal if the information exchange satisfies a certain safety condition. We then construct concrete programs that implement this knowledge-based program in two settings of interest that are shown to satisfy the safety condition. Finally, we show that a small modification of our program results in an FIP that is both optimal and efficiently implementable, settling an open problem posed by Halpern, Moses, and Waarts (SIAM J. Comput., 2001). △ Less

Submitted 10 May, 2023; originally announced May 2023.

Comments: The full version of the PODC 2023 paper

Joint Behavior and Common Belief

Authors: Meir Friedenberg, Joseph Y. Halpern

Abstract: For over 25 years, common belief has been widely viewed as necessary for joint behavior. But this is not quite correct. We show by example that what can naturally be thought of as joint behavior can occur without common belief. We then present two variants of common belief that can lead to joint behavior, even without standard common belief ever being achieved, and show that one of them, action-st… ▽ More For over 25 years, common belief has been widely viewed as necessary for joint behavior. But this is not quite correct. We show by example that what can naturally be thought of as joint behavior can occur without common belief. We then present two variants of common belief that can lead to joint behavior, even without standard common belief ever being achieved, and show that one of them, action-stamped common belief, is in a sense necessary and sufficient for joint behavior. These observations are significant because, as is well known, common belief is quite difficult to achieve in practice, whereas these variants are more easily achievable. △ Less

Submitted 11 July, 2023; v1 submitted 13 March, 2023; originally announced March 2023.

Comments: In Proceedings TARK 2023, arXiv:2307.04005

Journal ref: EPTCS 379, 2023, pp. 221-232

Causal Models with Constraints

Authors: Sander Beckers, Joseph Y. Halpern, Christopher Hitchcock

Abstract: Causal models have proven extremely useful in offering formal representations of causal relationships between a set of variables. Yet in many situations, there are non-causal relationships among variables. For example, we may want variables $LDL$, $HDL$, and $TOT$ that represent the level of low-density lipoprotein cholesterol, the level of lipoprotein high-density lipoprotein cholesterol, and tot… ▽ More Causal models have proven extremely useful in offering formal representations of causal relationships between a set of variables. Yet in many situations, there are non-causal relationships among variables. For example, we may want variables $LDL$, $HDL$, and $TOT$ that represent the level of low-density lipoprotein cholesterol, the level of lipoprotein high-density lipoprotein cholesterol, and total cholesterol level, with the relation $LDL+HDL=TOT$. This cannot be done in standard causal models, because we can intervene simultaneously on all three variables. The goal of this paper is to extend standard causal models to allow for constraints on settings of variables. Although the extension is relatively straightforward, to make it useful we have to define a new intervention operation that $disconnects$ a variable from a causal equation. We give examples showing the usefulness of this extension, and provide a sound and complete axiomatization for causal models with constraints. △ Less

Submitted 17 January, 2023; originally announced January 2023.

Comments: Accepted at CLeaR 2023

A Causal Analysis of Harm

Authors: Sander Beckers, Hana Chockler, Joseph Y. Halpern

Abstract: As autonomous systems rapidly become ubiquitous, there is a growing need for a legal and regulatory framework to address when and how such a system harms someone. There have been several attempts within the philosophy literature to define harm, but none of them has proven capable of dealing with with the many examples that have been presented, leading some to suggest that the notion of harm should… ▽ More As autonomous systems rapidly become ubiquitous, there is a growing need for a legal and regulatory framework to address when and how such a system harms someone. There have been several attempts within the philosophy literature to define harm, but none of them has proven capable of dealing with with the many examples that have been presented, leading some to suggest that the notion of harm should be abandoned and "replaced by more well-behaved notions". As harm is generally something that is caused, most of these definitions have involved causality at some level. Yet surprisingly, none of them makes use of causal models and the definitions of actual causality that they can express. In this paper we formally define a qualitative notion of harm that uses causal models and is based on a well-known definition of actual causality (Halpern, 2016). The key novelty of our definition is that it is based on contrastive causation and uses a default utility to which the utility of actual outcomes is compared. We show that our definition is able to handle the examples from the literature, and illustrate its importance for reasoning about situations involving autonomous systems. △ Less

Submitted 19 January, 2023; v1 submitted 11 October, 2022; originally announced October 2022.

Comments: Accepted at NeurIPS 2022

Quantifying Harm

Authors: Sander Beckers, Hana Chockler, Joseph Y. Halpern

Abstract: In a companion paper (Beckers et al. 2022), we defined a qualitative notion of harm: either harm is caused, or it is not. For practical applications, we often need to quantify harm; for example, we may want to choose the lest harmful of a set of possible interventions. We first present a quantitative definition of harm in a deterministic context involving a single individual, then we consider the… ▽ More In a companion paper (Beckers et al. 2022), we defined a qualitative notion of harm: either harm is caused, or it is not. For practical applications, we often need to quantify harm; for example, we may want to choose the lest harmful of a set of possible interventions. We first present a quantitative definition of harm in a deterministic context involving a single individual, then we consider the issues involved in dealing with uncertainty regarding the context and going from a notion of harm for a single individual to a notion of "societal harm", which involves aggregating the harm to individuals. We show that the "obvious" way of doing this (just taking the expected harm for an individual and then summing the expected harm over all individuals can lead to counterintuitive or inappropriate answers, and discuss alternatives, drawing on work from the decision-theory literature. △ Less

Submitted 6 October, 2022; v1 submitted 29 September, 2022; originally announced September 2022.

Comments: 17 pages, under submission

From Outcome-Based to Language-Based Preferences

Authors: Valerio Capraro, Joseph Y. Halpern, Matjaz Perc

Abstract: We review the literature on models that try to explain human behavior in social interactions described by normal-form games with monetary payoffs. We start by covering social and moral preferences. We then focus on the growing body of research showing that people react to the language in which actions are described, especially when it activates moral concerns. We conclude by arguing that behaviora… ▽ More We review the literature on models that try to explain human behavior in social interactions described by normal-form games with monetary payoffs. We start by covering social and moral preferences. We then focus on the growing body of research showing that people react to the language in which actions are described, especially when it activates moral concerns. We conclude by arguing that behavioral economics is in the midst of a paradigm shift towards language-based preferences, which will require an exploration of new models and experimental setups. △ Less

Submitted 15 June, 2022; originally announced June 2022.

Comments: Forthcoming in the Journal of Economic Literature

Journal ref: J. Econ. Lit. 62, 115-154 (2024)

The brain as a probabilistic transducer: an evolutionarily plausible network architecture for knowledge representation, computation, and behavior

Authors: Joseph Y. Halpern, Arnon Lotem

Abstract: We offer a general theoretical framework for brain and behavior that is evolutionarily and computationally plausible. The brain in our abstract model is a network of nodes and edges. Although it has some similarities to standard neural network models, as we show, there are some significant differences. Both nodes and edges in our network have weights and activation levels. They act as probabilisti… ▽ More We offer a general theoretical framework for brain and behavior that is evolutionarily and computationally plausible. The brain in our abstract model is a network of nodes and edges. Although it has some similarities to standard neural network models, as we show, there are some significant differences. Both nodes and edges in our network have weights and activation levels. They act as probabilistic transducers that use a set of relatively simple rules to determine how activation levels and weights are affected by input, generate output, and affect each other. We show that these simple rules enable a learning process that allows the network to represent increasingly complex knowledge, and simultaneously to act as a computing device that facilitates planning, decision-making, and the execution of behavior. By specifying the innate (genetic) components of the network, we show how evolution could endow the network with initial adaptive rules and goals that are then enriched through learning. We demonstrate how the developing structure of the network (which determines what the brain can do and how well) is critically affected by the co-evolved coordination between the mechanisms affecting the distribution of data input and those determining the learning parameters (used in the programs run by nodes and edges). Finally, we consider how the model accounts for various findings in the field of learning and decision making, how it can address some challenging problems in mind and behavior, such as those related to setting goals and self-control, and how it can help understand some cognitive disorders. △ Less

Submitted 11 April, 2022; v1 submitted 26 December, 2021; originally announced December 2021.

Reasoning About Causal Models With Infinitely Many Variables

Authors: Joseph Y. Halpern, Spencer Peters

Abstract: Generalized structural equations models (GSEMs) [Peters and Halpern 2021], are, as the name suggests, a generalization of structural equations models (SEMs). They can deal with (among other things) infinitely many variables with infinite ranges, which is critical for capturing dynamical systems. We provide a sound and complete axiomatization of causal reasoning in GSEMs that is an extension of the… ▽ More Generalized structural equations models (GSEMs) [Peters and Halpern 2021], are, as the name suggests, a generalization of structural equations models (SEMs). They can deal with (among other things) infinitely many variables with infinite ranges, which is critical for capturing dynamical systems. We provide a sound and complete axiomatization of causal reasoning in GSEMs that is an extension of the sound and complete axiomatization provided by Halpern [2000] for SEMs. Considering GSEMs helps clarify what properties Halpern's axioms capture. △ Less

Submitted 21 December, 2021; originally announced December 2021.

Causal Modeling With Infinitely Many Variables

Authors: Spencer Peters, Joseph Y. Halpern

Abstract: Structural-equations models (SEMs) are perhaps the most commonly used framework for modeling causality. However, as we show, naively extending this framework to infinitely many variables, which is necessary, for example, to model dynamical systems, runs into several problems. We introduce GSEMs (generalized SEMs), a flexible generalization of SEMs that directly specify the results of interventions… ▽ More Structural-equations models (SEMs) are perhaps the most commonly used framework for modeling causality. However, as we show, naively extending this framework to infinitely many variables, which is necessary, for example, to model dynamical systems, runs into several problems. We introduce GSEMs (generalized SEMs), a flexible generalization of SEMs that directly specify the results of interventions, in which (1) systems of differential equations can be represented in a natural and intuitive manner, (2) certain natural situations, which cannot be represented by SEMs at all, can be represented easily, (3) the definition of actual causality in SEMs carries over essentially without change. △ Less

Submitted 16 December, 2021; originally announced December 2021.

Tracking Truth with Liquid Democracy

Authors: Adam Berinsky, Daniel Halpern, Joseph Y. Halpern, Ali Jadbabaie, Elchanan Mossel, Ariel D. Procaccia, Manon Revel

Abstract: The dynamics of random transitive delegations on a graph are of particular interest when viewed through the lens of an emerging voting paradigm, liquid democracy. This paradigm allows voters to choose between directly voting and transitively delegating their votes to other voters, so that those selected cast a vote weighted by the number of delegations they received. In the epistemic setting, wher… ▽ More The dynamics of random transitive delegations on a graph are of particular interest when viewed through the lens of an emerging voting paradigm, liquid democracy. This paradigm allows voters to choose between directly voting and transitively delegating their votes to other voters, so that those selected cast a vote weighted by the number of delegations they received. In the epistemic setting, where voters decide on a binary issue for which there is a ground truth, previous work showed that a few voters may amass such a large amount of influence that liquid democracy is less likely to identify the ground truth than direct voting. We quantify the amount of permissible concentration of power and examine more realistic delegation models, showing they behave well by ensuring that (with high probability) there is a permissible limit on the maximum number of delegations received. Our theoretical results demonstrate that the delegation process is similar to well-known processes on random graphs that are sufficiently bounded for our purposes. Along the way, we prove new bounds on the size of the largest component in an infinite Pólya urn process, which may be of independent interest. In addition, we empirically validate the theoretical results, running six experiments (for a total of $N=168$ participants, $62$ delegation graphs and over $11k$ votes collected). We find that empirical delegation behaviors meet the conditions for our positive theoretical guarantees. Overall, our work alleviates concerns raised about liquid democracy and bolsters the case for the applicability of this emerging paradigm. △ Less

Submitted 23 August, 2024; v1 submitted 25 July, 2021; originally announced July 2021.

Comments: This version supersedes In Defense of Liquid Democracy by Halpern et. al which appeared in the Proceedings of the 24th ACM Conference on Economics and Computation (EC'23) as an extended abstract (https://dl.acm.org/doi/10.1145/3580507.3597817). This version adds experiments to the theoretical results of Halpern et. al

Language-based Decisions

Authors: Adam Bjorndahl, Joseph Y. Halpern

Abstract: In Savage's classic decision-theoretic framework, actions are formally defined as functions from states to outcomes. But where do the state space and outcome space come from? Expanding on recent work by Blume, Easley, and Halpern (BEH), we consider a language-based framework in which actions are identified with (conditional) descriptions in a simple underlying language, while states and outcomes (… ▽ More In Savage's classic decision-theoretic framework, actions are formally defined as functions from states to outcomes. But where do the state space and outcome space come from? Expanding on recent work by Blume, Easley, and Halpern (BEH), we consider a language-based framework in which actions are identified with (conditional) descriptions in a simple underlying language, while states and outcomes (along with probabilities and utilities) are constructed as part of a representation theorem. Our work expands the role of language from that of BEH by using it not only for the conditions that determine which actions are taken, but also the effects. More precisely, we take the set of actions to be built from those of the form "do(phi)", for formulas phi in the underlying language. This presents a problem: how do we interpret the result of do(phi) when phi is underspecified (i.e., compatible with multiple states)? We answer this using tools familiar from the semantics of counterfactuals: roughly speaking, do(phi) maps each state to the "closest" phi-state. This notion of "closest" is also something we construct as part of the representation theorem; in effect, then, we prove that (under appropriate assumptions) the agent is acting as if each underspecified action is first made definite and then evaluated (i.e., by maximizing expected utility). Of course, actions in the real world are often not presented in a fully precise manner, yet agents reason about and form preferences among them all the same. Our work brings the abstract tools of decision theory into closer contact with such real-world scenarios. △ Less

Submitted 21 June, 2021; originally announced June 2021.

Comments: In Proceedings TARK 2021, arXiv:2106.10886

Journal ref: EPTCS 335, 2021, pp. 55-67

Proceedings Eighteenth Conference on Theoretical Aspects of Rationality and Knowledge

Authors: Joseph Halpern, Andrés Perea

Abstract: The TARK conference (Theoretical Aspects of Rationality and Knowledge) is a biannual conference that aims to bring together researchers from a wide variety of fields, including computer science, artificial intelligence, game theory, decision theory, philosophy, logic, linguistics, and cognitive science. Its goal is to further our understanding of interdisciplinary issues involving reasoning about… ▽ More The TARK conference (Theoretical Aspects of Rationality and Knowledge) is a biannual conference that aims to bring together researchers from a wide variety of fields, including computer science, artificial intelligence, game theory, decision theory, philosophy, logic, linguistics, and cognitive science. Its goal is to further our understanding of interdisciplinary issues involving reasoning about rationality and knowledge. Topics of interest include, but are not limited to, semantic models for knowledge, belief, awareness and uncertainty, bounded rationality and resource-bounded reasoning, commonsense epistemic reasoning, epistemic logic, epistemic game theory, knowledge and action, applications of reasoning about knowledge and other mental states, belief revision, and foundations of multi-agent systems. These proceedings contain the papers that have been accepted for presentation at the Eighteenth Conference on Theoretical Aspects of Rationality and Knowledge (TARK 2021), held between June 25 and June 27, 2021, at Tsinghua University at Beijing, China. △ Less

Submitted 21 June, 2021; originally announced June 2021.

Journal ref: EPTCS 335, 2021

Lower Bounds Implementing Mediators in Asynchronous Systems

Authors: Ivan Geffner, Joseph Y. Halpern

Abstract: Abraham, Dolev, Geffner, and Halpern proved that, in asynchronous systems, a $(k,t)$-robust equilibrium for $n$ players and a trusted mediator can be implemented without the mediator as long as $n > 4(k+t)$, where an equilibrium is $(k,t)$-robust if, roughly speaking, no coalition of $t$ players can decrease the payoff of any of the other players, and no coalition of $k$ players can increase their… ▽ More Abraham, Dolev, Geffner, and Halpern proved that, in asynchronous systems, a $(k,t)$-robust equilibrium for $n$ players and a trusted mediator can be implemented without the mediator as long as $n > 4(k+t)$, where an equilibrium is $(k,t)$-robust if, roughly speaking, no coalition of $t$ players can decrease the payoff of any of the other players, and no coalition of $k$ players can increase their payoff by deviating. We prove that this bound is tight, in the sense that if $n \le 4(k+t)$ there exist $(k,t)$-robust equilibria with a mediator that cannot be implemented by the players alone. Even though implementing $(k,t)$-robust mediators seems closely related to implementing asynchronous multiparty $(k+t)$-secure computation \cite{BCG93}, to the best of our knowledge there is no known straightforward reduction from one problem to another. Nevertheless, we show that there is a non-trivial reduction from a slightly weaker notion of $(k+t)$-secure computation, which we call $(k+t)$-strict secure computation, to implementing $(k,t)$-robust mediators. We prove the desired lower bound by showing that there are functions on $n$ variables that cannot be $(k+t)$-strictly securely computed if $n \le 4(k+t)$. This also provides a simple alternative proof for the well-known lower bound of $4t+1$ on asynchronous secure computation in the presence of up to $t$ malicious agents. △ Less

Submitted 6 April, 2021; originally announced April 2021.

Security Properties as Nested Causal Statements

Authors: Matvey Soloviev, Joseph Y. Halpern

Abstract: Thinking in terms of causality helps us structure how different parts of a system depend on each other, and how interventions on one part of a system may result in changes to other parts. Therefore, formal models of causality are an attractive tool for reasoning about security, which concerns itself with safeguarding properties of a system against interventions that may be malicious. As we show, m… ▽ More Thinking in terms of causality helps us structure how different parts of a system depend on each other, and how interventions on one part of a system may result in changes to other parts. Therefore, formal models of causality are an attractive tool for reasoning about security, which concerns itself with safeguarding properties of a system against interventions that may be malicious. As we show, many security properties are naturally expressed as nested causal statements: not only do we consider what caused a particular undesirable effect, but we also consider what caused this causal relationship itself to hold. We present a natural way to extend the Halpern-Pearl (HP) framework for causality to capture such nested causal statements. This extension adds expressivity, enabling the HP framework to distinguish between causal scenarios that it could not previously naturally tell apart. We moreover revisit some design decisions of the HP framework that were made with non-nested causal statements in mind, such as the choice to treat specific values of causal variables as opposed to the variables themselves as causes, and may no longer be appropriate for nested ones. △ Less

Submitted 1 April, 2021; originally announced April 2021.

Comments: 8 pages

ACM Class: I.2.4; D.4.6

Probabilistic Dependency Graphs

Authors: Oliver Richardson, Joseph Y Halpern

Abstract: We introduce Probabilistic Dependency Graphs (PDGs), a new class of directed graphical models. PDGs can capture inconsistent beliefs in a natural way and are more modular than Bayesian Networks (BNs), in that they make it easier to incorporate new information and restructure the representation. We show by example how PDGs are an especially natural modeling tool. We provide three semantics for PDGs… ▽ More We introduce Probabilistic Dependency Graphs (PDGs), a new class of directed graphical models. PDGs can capture inconsistent beliefs in a natural way and are more modular than Bayesian Networks (BNs), in that they make it easier to incorporate new information and restructure the representation. We show by example how PDGs are an especially natural modeling tool. We provide three semantics for PDGs, each of which can be derived from a scoring function (on joint distributions over the variables in the network) that can be viewed as representing a distribution's incompatibility with the PDG. For the PDG corresponding to a BN, this function is uniquely minimized by the distribution the BN represents, showing that PDG semantics extend BN semantics. We show further that factor graphs and their exponential families can also be faithfully represented as PDGs, while there are significant barriers to modeling a PDG with a factor graph. △ Less

Submitted 19 December, 2020; originally announced December 2020.

Comments: 5 figures, 7 pages

Dynamic Awareness

Authors: Joseph Y. Halpern, Evan Piermont

Abstract: We investigate how to model the beliefs of an agent who becomes more aware. We use the framework of Halpern and Rego (2013) by adding probability, and define a notion of a model transition that describes constraints on how, if an agent becomes aware of a new formula $φ$ in state $s$ of a model $M$, she transitions to state $s^*$ in a model $M^*$. We then discuss how such a model can be applied to… ▽ More We investigate how to model the beliefs of an agent who becomes more aware. We use the framework of Halpern and Rego (2013) by adding probability, and define a notion of a model transition that describes constraints on how, if an agent becomes aware of a new formula $φ$ in state $s$ of a model $M$, she transitions to state $s^*$ in a model $M^*$. We then discuss how such a model can be applied to information disclosure. △ Less

Submitted 6 July, 2020; originally announced July 2020.

Comments: To appear in the 17th International Conference on Principles of Knowledge Representation and Reasoning

Bounded Rationality in Las Vegas: Probabilistic Finite Automata PlayMulti-Armed Bandits

Authors: Xinming Liu, Joseph Y. Halpern

Abstract: While traditional economics assumes that humans are fully rational agents who always maximize their expected utility, in practice, we constantly observe apparently irrational behavior. One explanation is that people have limited computational power, so that they are, quite rationally, making the best decisions they can, given their computational limitations. To test this hypothesis, we consider th… ▽ More While traditional economics assumes that humans are fully rational agents who always maximize their expected utility, in practice, we constantly observe apparently irrational behavior. One explanation is that people have limited computational power, so that they are, quite rationally, making the best decisions they can, given their computational limitations. To test this hypothesis, we consider the multi-armed bandit (MAB) problem. We examine a simple strategy for playing an MAB that can be implemented easily by a probabilistic finite automaton (PFA). Roughly speaking, the PFA sets certain expectations, and plays an arm as long as it meets them. If the PFA has sufficiently many states, it performs near-optimally. Its performance degrades gracefully as the number of states decreases. Moreover, the PFA acts in a "human-like" way, exhibiting a number of standard human biases, like an optimism bias and a negativity bias. △ Less

Submitted 30 June, 2020; originally announced June 2020.

Comments: 10 pages, 4 pages, Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI), PMLR volume 124, 2020

Information Acquisition Under Resource Limitations in a Noisy Environment

Authors: Matvey Soloviev, Joseph Y. Halpern

Abstract: We introduce a theoretical model of information acquisition under resource limitations in a noisy environment. An agent must guess the truth value of a given Boolean formula $\varphi$ after performing a bounded number of noisy tests of the truth values of variables in the formula. We observe that, in general, the problem of finding an optimal testing strategy for $φ$ is hard, but we suggest a usef… ▽ More We introduce a theoretical model of information acquisition under resource limitations in a noisy environment. An agent must guess the truth value of a given Boolean formula $\varphi$ after performing a bounded number of noisy tests of the truth values of variables in the formula. We observe that, in general, the problem of finding an optimal testing strategy for $φ$ is hard, but we suggest a useful heuristic. The techniques we use also give insight into two apparently unrelated, but well-studied problems: (1) \emph{rational inattention}, that is, when it is rational to ignore pertinent information (the optimal strategy may involve hardly ever testing variables that are clearly relevant to $φ$), and (2) what makes a formula hard to learn/remember. △ Less

Submitted 20 May, 2020; originally announced May 2020.

Comments: A preliminary version of the paper appeared in \emph{Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18)}, 2018

MDPs with Unawareness in Robotics

Authors: Nan Rong, Joseph Y. Halpern, Ashutosh Saxena

Abstract: We formalize decision-making problems in robotics and automated control using continuous MDPs and actions that take place over continuous time intervals. We then approximate the continuous MDP using finer and finer discretizations. Doing this results in a family of systems, each of which has an extremely large action space, although only a few actions are "interesting". We can view the decision ma… ▽ More We formalize decision-making problems in robotics and automated control using continuous MDPs and actions that take place over continuous time intervals. We then approximate the continuous MDP using finer and finer discretizations. Doing this results in a family of systems, each of which has an extremely large action space, although only a few actions are "interesting". We can view the decision maker as being unaware of which actions are "interesting". We can model this using MDPUs, MDPs with unawareness, where the action space is much smaller. As we show, MDPUs can be used as a general framework for learning tasks in robotic problems. We prove results on the difficulty of learning a near-optimal policy in an an MDPU for a continuous task. We apply these ideas to the problem of having a humanoid robot learn on its own how to walk. △ Less

Submitted 20 May, 2020; originally announced May 2020.

Comments: Appears in Proceedings of the 32nd Conference on Uncertainty in AI (UAI 2016), 2016

Causality, Responsibility and Blame in Team Plans

Authors: Natasha Alechina, Joseph Y. Halpern, Brian Logan

Abstract: Many objectives can be achieved (or may be achieved more effectively) only by a group of agents executing a team plan. If a team plan fails, it is often of interest to determine what caused the failure, the degree of responsibility of each agent for the failure, and the degree of blame attached to each agent. We show how team plans can be represented in terms of structural equations, and then appl… ▽ More Many objectives can be achieved (or may be achieved more effectively) only by a group of agents executing a team plan. If a team plan fails, it is often of interest to determine what caused the failure, the degree of responsibility of each agent for the failure, and the degree of blame attached to each agent. We show how team plans can be represented in terms of structural equations, and then apply the definitions of causality introduced by Halpern [2015] and degree of responsibility and blame introduced by Chockler and Halpern [2004] to determine the agent(s) who caused the failure and what their degree of responsibility/blame is. We also prove new results on the complexity of computing causality and degree of responsibility and blame, showing that they can be determined in polynomial time for many team plans of interest. △ Less

Submitted 20 May, 2020; originally announced May 2020.

Comments: {\em Proceedings of the Sixteenth Appears in \emph{Proceedings of the International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS 2017)}, 2017

Combining Experts' Causal Judgments

Authors: Dalal Alrajeh, Hana Chockler, Joseph Y. Halpern

Abstract: Consider a policymaker who wants to decide which intervention to perform in order to change a currently undesirable situation. The policymaker has at her disposal a team of experts, each with their own understanding of the causal dependencies between different factors contributing to the outcome. The policymaker has varying degrees of confidence in the experts' opinions. She wants to combine their… ▽ More Consider a policymaker who wants to decide which intervention to perform in order to change a currently undesirable situation. The policymaker has at her disposal a team of experts, each with their own understanding of the causal dependencies between different factors contributing to the outcome. The policymaker has varying degrees of confidence in the experts' opinions. She wants to combine their opinions in order to decide on the most effective intervention. We formally define the notion of an effective intervention, and then consider how experts' causal judgments can be combined in order to determine the most effective intervention. We define a notion of two causal models being \emph{compatible}, and show how compatible causal models can be merged. We then use it as the basis for combining experts' causal judgments. We also provide a definition of decomposition for causal models to cater for cases when models are incompatible. We illustrate our approach on a number of real-life examples. △ Less

Submitted 20 May, 2020; originally announced May 2020.

Comments: A preliminary version of the paper appeared in \emph{Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence (AAAI-18)}, 2018}

Rational Consensus

Authors: Joseph Y. Halpern, Xavier Vilaca

Abstract: We provide a game-theoretic analysis of consensus, assuming that processes are controlled by rational agents and may fail by crashing. We consider agents that \emph{care only about consensus}: that is, (a) an agent's utility depends only on the consensus value achieved (and not, for example, on the number of messages the agent sends) and (b) agents strictly prefer reaching consensus to not reachin… ▽ More We provide a game-theoretic analysis of consensus, assuming that processes are controlled by rational agents and may fail by crashing. We consider agents that \emph{care only about consensus}: that is, (a) an agent's utility depends only on the consensus value achieved (and not, for example, on the number of messages the agent sends) and (b) agents strictly prefer reaching consensus to not reaching consensus. We show that, under these assumptions, there is no \emph{ex post Nash Equilibrium}, even with only one failure. Roughly speaking, this means that there must always exist a \emph{failure pattern} (a description of who fails, when they fail, and which agents they do not send messages to in the round that they fail) and initial preferences for which an agent can gain by deviating. On the other hand, if we assume that there is a distribution $π$ on the failure patterns and initial preferences, then under minimal assumptions on $π$, there is a Nash equilibrium that tolerates $f$ failures (i.e., $π$ puts probability 1 on there being at most $f$ failures) if $f+1 < n$ (where $n$ is the total number of agents). Moreover, we show that a slight extension of the Nash equilibrium strategy is also a \emph{sequential} equilibrium (under the same assumptions about the distribution $π$). △ Less

Submitted 20 May, 2020; originally announced May 2020.

Comments: Appears in Proceedings of the 35th Annual ACM Symposium on Principles of Distributed Computing, 2016

Combining the Causal Judgments of Experts with Possibly Different Focus Areas

Authors: Meir Friedenberg, Joseph Y. Halpern

Abstract: In many real-world settings, a decision-maker must combine information provided by different experts in order to decide on an effective policy. Alrajeh, Chockler, and Halpern [2018] showed how to combine causal models that are compatible in the sense that, for variables that appear in both models, the experts agree on the causal structure. In this work we show how causal models can be combined in… ▽ More In many real-world settings, a decision-maker must combine information provided by different experts in order to decide on an effective policy. Alrajeh, Chockler, and Halpern [2018] showed how to combine causal models that are compatible in the sense that, for variables that appear in both models, the experts agree on the causal structure. In this work we show how causal models can be combined in cases where the experts might disagree on the causal structure for variables that appear in both models due to having different focus areas. We provide a new formal definition of compatibility of models in this setting and show how compatible models can be combined. We also consider the complexity of determining whether models are compatible. We believe that the notions defined in this work are of direct relevance to many practical decision making scenarios that come up in natural, social, and medical science settings. △ Less

Submitted 20 May, 2020; originally announced May 2020.

Comments: Appear in the Proceedings of the Sixteenth International Conference on Principles of Knowledge Representation and Reasoning (KR2018}, 2018

The Book of Why: Review

Authors: Joseph Y. Halpern

Abstract: This is a review of "The Book of Why", by Judea Pearl. This is a review of "The Book of Why", by Judea Pearl. △ Less

Submitted 30 September, 2019; originally announced September 2019.

Comments: To appear in "Artificial Intelligence" journal

A Conceptually Well-Founded Characterization of Iterated Admissibility Using an "All I Know" Operator

Authors: Joseph Y. Halpern, Rafael Pass

Abstract: Brandenburger, Friedenberg, and Keisler provide an epistemic characterization of iterated admissibility (IA), also known as iterated deletion of weakly dominated strategies, where uncertainty is represented using LPSs (lexicographic probability sequences). Their characterization holds in a rich structure called a complete structure, where all types are possible. In earlier work, we gave a characte… ▽ More Brandenburger, Friedenberg, and Keisler provide an epistemic characterization of iterated admissibility (IA), also known as iterated deletion of weakly dominated strategies, where uncertainty is represented using LPSs (lexicographic probability sequences). Their characterization holds in a rich structure called a complete structure, where all types are possible. In earlier work, we gave a characterization of iterated admissibility using an "all I know" operator, that captures the intuition that "all the agent knows" is that agents satisfy the appropriate rationality assumptions. That characterization did not need complete structures and used probability structures, not LPSs. However, that characterization did not deal with Samuelson's conceptual concern regarding IA, namely, that at higher levels, players do not consider possible strategies that were used to justify their choice of strategy at lower levels. In this paper, we give a characterization of IA using the all I know operator that does deal with Samuelson's concern. However, it uses LPSs. We then show how to modify the characterization using notions of "approximate belief" and "approximately all I know" so as to deal with Samuelson's concern while still working with probability structures. △ Less

Submitted 21 July, 2019; originally announced July 2019.

Comments: In Proceedings TARK 2019, arXiv:1907.08335

Journal ref: EPTCS 297, 2019, pp. 221-232

Approximate Causal Abstraction

Authors: Sander Beckers, Frederick Eberhardt, Joseph Y. Halpern

Abstract: Scientific models describe natural phenomena at different levels of abstraction. Abstract descriptions can provide the basis for interventions on the system and explanation of observed phenomena at a level of granularity that is coarser than the most fundamental account of the system. Beckers and Halpern (2019), building on work of Rubenstein et al. (2017), developed an account of abstraction for… ▽ More Scientific models describe natural phenomena at different levels of abstraction. Abstract descriptions can provide the basis for interventions on the system and explanation of observed phenomena at a level of granularity that is coarser than the most fundamental account of the system. Beckers and Halpern (2019), building on work of Rubenstein et al. (2017), developed an account of abstraction for causal models that is exact. Here we extend this account to the more realistic case where an abstract causal model offers only an approximation of the underlying system. We show how the resulting account handles the discrepancy that can arise between low- and high-level causal models of the same system, and in the process provide an account of how one causal model approximates another, a topic of independent interest. Finally, we extend the account of approximate abstractions to probabilistic causal models, indicating how and where uncertainty can enter into an approximate abstraction. △ Less

Submitted 29 June, 2019; v1 submitted 27 June, 2019; originally announced June 2019.

Comments: Appears in UAI-2019

Security in Asynchronous Interactive Systems

Authors: Ivan Geffner, Joseph Y. Halpern

Abstract: Secure function computation has been thoroughly studied and optimized in the past decades. We extend techniques used for secure computation to simulate arbitrary protocols involving a mediator. The key feature of our notion of simulation is that it is bidirectional: not only does the simulation produce only outputs that could happen in the original protocol, but the simulation produces all such ou… ▽ More Secure function computation has been thoroughly studied and optimized in the past decades. We extend techniques used for secure computation to simulate arbitrary protocols involving a mediator. The key feature of our notion of simulation is that it is bidirectional: not only does the simulation produce only outputs that could happen in the original protocol, but the simulation produces all such outputs. In a synchronous system, it can be shown that this requirement can already be achieved by the standard notion of secure computation. However, in an asynchronous system, new subtleties arise because the scheduler can influence the output. We provide a construction that is secure if $n > 4t$, where $t$ is the number malicious agents, which is provably the best possible. We also show that our construction satisfies additional security properties even if $3t < n \le 4t$. △ Less

Submitted 5 June, 2019; originally announced June 2019.

ACM Class: C.2.4

Blameworthiness in Multi-Agent Settings

Authors: Meir Friedenberg, Joseph Y. Halpern

Abstract: We provide a formal definition of blameworthiness in settings where multiple agents can collaborate to avoid a negative outcome. We first provide a method for ascribing blameworthiness to groups relative to an epistemic state (a distribution over causal models that describe how the outcome might arise). We then show how we can go from an ascription of blameworthiness for groups to an ascription of… ▽ More We provide a formal definition of blameworthiness in settings where multiple agents can collaborate to avoid a negative outcome. We first provide a method for ascribing blameworthiness to groups relative to an epistemic state (a distribution over causal models that describe how the outcome might arise). We then show how we can go from an ascription of blameworthiness for groups to an ascription of blameworthiness for individuals using a standard notion from cooperative game theory, the Shapley value. We believe that getting a good notion of blameworthiness in a group setting will be critical for designing autonomous agents that behave in a moral manner. △ Less

Submitted 10 March, 2019; originally announced March 2019.

Comments: Appears in AAAI-19

Abstracting Causal Models

Authors: Sander Beckers, Joseph Y. Halpern

Abstract: We consider a sequence of successively more restrictive definitions of abstraction for causal models, starting with a notion introduced by Rubenstein et al. (2017) called exact transformation that applies to probabilistic causal models, moving to a notion of uniform transformation that applies to deterministic causal models and does not allow differences to be hidden by the "right" choice of distr… ▽ More We consider a sequence of successively more restrictive definitions of abstraction for causal models, starting with a notion introduced by Rubenstein et al. (2017) called exact transformation that applies to probabilistic causal models, moving to a notion of uniform transformation that applies to deterministic causal models and does not allow differences to be hidden by the "right" choice of distribution, and then to abstraction, where the interventions of interest are determined by the map from low-level states to high-level states, and strong abstraction, which takes more seriously all potential interventions in a model, not just the allowed interventions. We show that procedures for combining micro-variables into macro-variables are instances of our notion of strong abstraction, as are all the examples considered by Rubenstein et al. △ Less

Submitted 9 July, 2019; v1 submitted 10 December, 2018; originally announced December 2018.

Comments: Appears in AAAI-19

Partial Awareness

Authors: Joseph Y. Halpern, Evan Piermont

Abstract: We develop a modal logic to capture partial awareness. The logic has three building blocks: objects, properties, and concepts. Properties are unary predicates on objects; concepts are Boolean combinations of properties. We take an agent to be partially aware of a concept if she is aware of the concept without being aware of the properties that define it. The logic allows for quantification over ob… ▽ More We develop a modal logic to capture partial awareness. The logic has three building blocks: objects, properties, and concepts. Properties are unary predicates on objects; concepts are Boolean combinations of properties. We take an agent to be partially aware of a concept if she is aware of the concept without being aware of the properties that define it. The logic allows for quantification over objects and properties, so that the agent can reason about her own unawareness. We then apply the logic to contracts, which we view as syntactic objects that dictate outcomes based on the truth of formulas. We show that when agents are unaware of some relevant properties, referencing concepts that agents are only partially aware of can improve welfare. △ Less

Submitted 14 November, 2018; originally announced November 2018.

Comments: Appears in AAAI-19

Towards Formal Definitions of Blameworthiness, Intention, and Moral Responsibility

Authors: Joseph Y. Halpern, Max Kleiman-Weiner

Abstract: We provide formal definitions of degree of blameworthiness and intention relative to an epistemic state (a probability over causal models and a utility function on outcomes). These, together with a definition of actual causality, provide the key ingredients for moral responsibility judgments. We show that these definitions give insight into commonsense intuitions in a variety of puzzling cases fro… ▽ More We provide formal definitions of degree of blameworthiness and intention relative to an epistemic state (a probability over causal models and a utility function on outcomes). These, together with a definition of actual causality, provide the key ingredients for moral responsibility judgments. We show that these definitions give insight into commonsense intuitions in a variety of puzzling cases from the literature. △ Less

Submitted 13 October, 2018; originally announced October 2018.

Comments: Appears in AAAI-18

Implementing Mediators with Asynchronous Cheap Talk

Authors: Ittai Abraham, Danny Dolev, Ivan Geffner, Joseph Y. Halpern

Abstract: A mediator can help non-cooperative agents obtain an equilibrium that may otherwise not be possible. We study the ability of players to obtain the same equilibrium without a mediator, using only cheap talk, that is, nonbinding pre-play communication. Previous work has considered this problem in a synchronous setting. Here we consider the effect of asynchrony on the problem, and provide upper bound… ▽ More A mediator can help non-cooperative agents obtain an equilibrium that may otherwise not be possible. We study the ability of players to obtain the same equilibrium without a mediator, using only cheap talk, that is, nonbinding pre-play communication. Previous work has considered this problem in a synchronous setting. Here we consider the effect of asynchrony on the problem, and provide upper bounds for implementing mediators. Considering asynchronous environments introduces new subtleties, including exactly what solution concept is most appropriate and determining what move is played if the cheap talk goes on forever. Different results are obtained depending on whether the move after such "infinite play" is under the control of the players or part of the description of the game. △ Less

Submitted 4 June, 2018; originally announced June 2018.

A Knowledge-Based Analysis of the Blockchain Protocol

Authors: Joseph Y. Halpern, Rafael Pass

Abstract: At the heart of the Bitcoin is a blockchain protocol, a protocol for achieving consensus on a public ledger that records bitcoin transactions. To the extent that a blockchain protocol is used for applications such as contract signing and making certain transactions (such as house sales) public, we need to understand what guarantees the protocol gives us in terms of agents' knowledge. Here, we p… ▽ More At the heart of the Bitcoin is a blockchain protocol, a protocol for achieving consensus on a public ledger that records bitcoin transactions. To the extent that a blockchain protocol is used for applications such as contract signing and making certain transactions (such as house sales) public, we need to understand what guarantees the protocol gives us in terms of agents' knowledge. Here, we provide a complete characterization of agent's knowledge when running a blockchain protocol using a variant of common knowledge that takes into account the fact that agents can enter and leave the system, it is not known which agents are in fact following the protocol (some agents may want to deviate if they can gain by doing so), and the fact that the guarantees provided by blockchain protocols are probabilistic. We then consider some scenarios involving contracts and show that this level of knowledge suffices for some scenarios, but not others. △ Less

Submitted 27 July, 2017; originally announced July 2017.

Comments: In Proceedings TARK 2017, arXiv:1707.08250

Journal ref: EPTCS 251, 2017, pp. 324-335

An Epistemic Foundation for Authentication Logics (Extended Abstract)

Authors: Joseph Y. Halpern, Ron van der Meyden, Riccardo Pucella

Abstract: While there have been many attempts, going back to BAN logic, to base reasoning about security protocols on epistemic notions, they have not been all that successful. Arguably, this has been due to the particular logics chosen. We present a simple logic based on the well-understood modal operators of knowledge, time, and probability, and show that it is able to handle issues that have often been s… ▽ More While there have been many attempts, going back to BAN logic, to base reasoning about security protocols on epistemic notions, they have not been all that successful. Arguably, this has been due to the particular logics chosen. We present a simple logic based on the well-understood modal operators of knowledge, time, and probability, and show that it is able to handle issues that have often been swept under the rug by other approaches, while being flexible enough to capture all the higher- level security notions that appear in BAN logic. Moreover, while still assuming that the knowledge operator allows for unbounded computation, it can handle the fact that a computationally bounded agent cannot decrypt messages in a natural way, by distinguishing strings and message terms. We demonstrate that our logic can capture BAN logic notions by providing a translation of the BAN operators into our logic, capturing belief by a form of probabilistic knowledge. △ Less

Submitted 27 July, 2017; originally announced July 2017.

Comments: In Proceedings TARK 2017, arXiv:1707.08250

Journal ref: EPTCS 251, 2017, pp. 306-323

Games With Tolerant Players

Authors: Arpita Ghosh, Joseph Y. Halpern

Abstract: A notion of pi-tolerant equilibrium is defined that takes into account that players have some tolerance regarding payoffs in a game. This solution concept generalizes Nash and refines epsilon-Nash equilibrium in a natural way. We show that pi-tolerant equilibrium can explain cooperation in social dilemmas such as Prisoner's Dilemma and the Public Good game. We then examine the structure of part… ▽ More A notion of pi-tolerant equilibrium is defined that takes into account that players have some tolerance regarding payoffs in a game. This solution concept generalizes Nash and refines epsilon-Nash equilibrium in a natural way. We show that pi-tolerant equilibrium can explain cooperation in social dilemmas such as Prisoner's Dilemma and the Public Good game. We then examine the structure of particularly cooperative pi-tolerant equilibria, where players are as cooperative as they can be, subject to their tolerances, in Prisoner's Dilemma. To the extent that cooperation is due to tolerance, these results provide guidance to a mechanism designer who has some control over the payoffs in a game, and suggest ways in which cooperation can be increased. △ Less

Submitted 27 July, 2017; originally announced July 2017.

Comments: In Proceedings TARK 2017, arXiv:1707.08250

Journal ref: EPTCS 251, 2017, pp. 251-264

From Type Spaces to Probability Frames and Back, via Language

Authors: Adam Bjorndahl, Joseph Y. Halpern

Abstract: We investigate the connection between the two major mathematical frameworks for modeling interactive beliefs: Harsanyi type spaces and possible-worlds style probability frames. While translating the former into the latter is straightforward, we demonstrate that the reverse translation relies implicitly on a background logical language. Once this "language parameter" is made explicit, it reveals a… ▽ More We investigate the connection between the two major mathematical frameworks for modeling interactive beliefs: Harsanyi type spaces and possible-worlds style probability frames. While translating the former into the latter is straightforward, we demonstrate that the reverse translation relies implicitly on a background logical language. Once this "language parameter" is made explicit, it reveals a close relationship between universal type spaces and canonical models: namely, that they are essentially the same construct. As the nature of a canonical model depends heavily on the background logic used to generate it, this work suggests a new view into a corresponding landscape of universal type spaces. △ Less

Submitted 27 July, 2017; originally announced July 2017.

Comments: In Proceedings TARK 2017, arXiv:1707.08250

Journal ref: EPTCS 251, 2017, pp. 75-87