Showing 1–50 of 59 results for author: Jones, N

Achieving AoI Fairness in Spatially Distributed Wireless Networks: From Theory to Implementation

Authors: Nicholas Jones, Joshua Wornell, Chao Li, Eytan Modiano

Abstract: We design and implement two variants of a practical random access protocol called WiFair, based on IEEE 802.11 and designed to mitigate spatial unfairness in Age of Information (AoI). Drawing on previous theoretical work, we modify the mechanics of 802.11 to fairly minimize AoI in a wireless network consisting of several update nodes and a single base station. We implement this protocol on a testb… ▽ More We design and implement two variants of a practical random access protocol called WiFair, based on IEEE 802.11 and designed to mitigate spatial unfairness in Age of Information (AoI). Drawing on previous theoretical work, we modify the mechanics of 802.11 to fairly minimize AoI in a wireless network consisting of several update nodes and a single base station. We implement this protocol on a testbed of software defined radios (SDRs) and measure its performance under a variety of settings compared to standard 802.11. We observe a 32% reduction in network average AoI and an 89% reduction in peak AoI in a last come first served (LCFS) single-packet queue setting, as well as a 76% reduction in network average AoI and an 82% reduction in peak AoI in a first come first served (FCFS) queue setting when the network is congested. We further show that when the network is uncongested, WiFair achieves the same performance as 802.11, and we demonstrate its robustness to more bursty traffic by streaming live video. △ Less

Submitted 28 August, 2024; originally announced August 2024.

PATIENT-Ψ: Using Large Language Models to Simulate Patients for Training Mental Health Professionals

Authors: Ruiyi Wang, Stephanie Milani, Jamie C. Chiu, Jiayin Zhi, Shaun M. Eack, Travis Labrum, Samuel M. Murphy, Nev Jones, Kate Hardy, Hong Shen, Fei Fang, Zhiyu Zoey Chen

Abstract: Mental illness remains one of the most critical public health issues. Despite its importance, many mental health professionals highlight a disconnect between their training and actual real-world patient practice. To help bridge this gap, we propose PATIENT-Ψ, a novel patient simulation framework for cognitive behavior therapy (CBT) training. To build PATIENT-Ψ, we construct diverse patient cogniti… ▽ More Mental illness remains one of the most critical public health issues. Despite its importance, many mental health professionals highlight a disconnect between their training and actual real-world patient practice. To help bridge this gap, we propose PATIENT-Ψ, a novel patient simulation framework for cognitive behavior therapy (CBT) training. To build PATIENT-Ψ, we construct diverse patient cognitive models based on CBT principles and use large language models (LLMs) programmed with these cognitive models to act as a simulated therapy patient. We propose an interactive training scheme, PATIENT-Ψ-TRAINER, for mental health trainees to practice a key skill in CBT -- formulating the cognitive model of the patient -- through role-playing a therapy session with PATIENT-Ψ. To evaluate PATIENT-Ψ, we conducted a comprehensive user study of 13 mental health trainees and 20 experts. The results demonstrate that practice using PATIENT-Ψ-TRAINER enhances the perceived skill acquisition and confidence of the trainees beyond existing forms of training such as textbooks, videos, and role-play with non-patients. Based on the experts' perceptions, PATIENT-Ψ is perceived to be closer to real patient interactions than GPT-4, and PATIENT-Ψ-TRAINER holds strong promise to improve trainee competencies. Our code and data are released at \url{https://github.com/ruiyiw/patient-psi}. △ Less

Submitted 18 June, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

Comments: 9 pages, 5 figures

Optimal Slicing and Scheduling with Service Guarantees in Multi-Hop Wireless Networks

Authors: Nicholas Jones, Eytan Modiano

Abstract: We analyze the problem of scheduling in wireless networks to meet end-to-end service guarantees. Using network slicing to decouple the queueing dynamics between flows, we show that the network's ability to meet hard throughput and deadline requirements is largely influenced by the scheduling policy. We characterize the feasible throughput/deadline region for a flow under a fixed route and set of s… ▽ More We analyze the problem of scheduling in wireless networks to meet end-to-end service guarantees. Using network slicing to decouple the queueing dynamics between flows, we show that the network's ability to meet hard throughput and deadline requirements is largely influenced by the scheduling policy. We characterize the feasible throughput/deadline region for a flow under a fixed route and set of slices, and find throughput- and deadline-optimal policies for a solitary flow. We formulate the feasibility problem for multiple flows in a general topology as a mixed-integer program, and show that it grows exponentially in the size of the network. Drawing on results from the solitary flow setting, we show that scheduling links in a regular fashion leads to smaller delay, and we derive tighter upper bounds on end-to-end delay for regular schedules. Finally, we design a polynomial-time algorithm that returns an (almost) regular schedule, optimized to meet service guarantees for all flows. △ Less

Submitted 15 April, 2024; v1 submitted 12 April, 2024; originally announced April 2024.

The Promises and Pitfalls of Using Language Models to Measure Instruction Quality in Education

Authors: Paiheng Xu, Jing Liu, Nathan Jones, Julie Cohen, Wei Ai

Abstract: Assessing instruction quality is a fundamental component of any improvement efforts in the education system. However, traditional manual assessments are expensive, subjective, and heavily dependent on observers' expertise and idiosyncratic factors, preventing teachers from getting timely and frequent feedback. Different from prior research that mostly focuses on low-inference instructional practic… ▽ More Assessing instruction quality is a fundamental component of any improvement efforts in the education system. However, traditional manual assessments are expensive, subjective, and heavily dependent on observers' expertise and idiosyncratic factors, preventing teachers from getting timely and frequent feedback. Different from prior research that mostly focuses on low-inference instructional practices on a singular basis, this paper presents the first study that leverages Natural Language Processing (NLP) techniques to assess multiple high-inference instructional practices in two distinct educational settings: in-person K-12 classrooms and simulated performance tasks for pre-service teachers. This is also the first study that applies NLP to measure a teaching practice that is widely acknowledged to be particularly effective for students with special needs. We confront two challenges inherent in NLP-based instructional analysis, including noisy and long input data and highly skewed distributions of human ratings. Our results suggest that pretrained Language Models (PLMs) demonstrate performances comparable to the agreement level of human raters for variables that are more discrete and require lower inference, but their efficacy diminishes with more complex teaching practices. Interestingly, using only teachers' utterances as input yields strong results for student-centered variables, alleviating common concerns over the difficulty of collecting and transcribing high-quality student speech data in in-person teaching settings. Our findings highlight both the potential and the limitations of current NLP techniques in the education domain, opening avenues for further exploration. △ Less

Submitted 3 April, 2024; originally announced April 2024.

Comments: NAACL 2024

Principled Preferential Bayesian Optimization

Authors: Wenjie Xu, Wenbin Wang, Yuning Jiang, Bratislav Svetozarevic, Colin N. Jones

Abstract: We study the problem of preferential Bayesian optimization (BO), where we aim to optimize a black-box function with only preference feedback over a pair of candidate solutions. Inspired by the likelihood ratio idea, we construct a confidence set of the black-box function using only the preference feedback. An optimistic algorithm with an efficient computational method is then developed to solve th… ▽ More We study the problem of preferential Bayesian optimization (BO), where we aim to optimize a black-box function with only preference feedback over a pair of candidate solutions. Inspired by the likelihood ratio idea, we construct a confidence set of the black-box function using only the preference feedback. An optimistic algorithm with an efficient computational method is then developed to solve the problem, which enjoys an information-theoretic bound on the total cumulative regret, a first-of-its-kind for preferential BO. This bound further allows us to design a scheme to report an estimated best solution, with a guaranteed convergence rate. Experimental results on sampled instances from Gaussian processes, standard test functions, and a thermal comfort optimization problem all show that our method stably achieves better or competitive performance as compared to the existing state-of-the-art heuristics, which, however, do not have theoretical guarantees on regret bounds or convergence. △ Less

Submitted 29 May, 2024; v1 submitted 7 February, 2024; originally announced February 2024.

Comments: Accepted to ICML 2024

Ensuring Data Privacy in AC Optimal Power Flow with a Distributed Co-Simulation Framework

Authors: Xinliang Dai, Alexander Kocher, Jovana Kovačević, Burak Dindar, Yuning Jiang, Colin N. Jones, Hüseyin Çakmak, Veit Hagenmeyer

Abstract: During the energy transition, the significance of collaborative management among institutions is rising, confronting challenges posed by data privacy concerns. Prevailing research on distributed approaches, as an alternative to centralized management, often lacks numerical convergence guarantees or is limited to single-machine numerical simulation. To address this, we present a distributed approac… ▽ More During the energy transition, the significance of collaborative management among institutions is rising, confronting challenges posed by data privacy concerns. Prevailing research on distributed approaches, as an alternative to centralized management, often lacks numerical convergence guarantees or is limited to single-machine numerical simulation. To address this, we present a distributed approach for solving AC Optimal Power Flow (OPF) problems within a geographically distributed environment. This involves integrating the energy system Co-Simulation (eCoSim) module in the eASiMOV framework with the convergence-guaranteed distributed optimization algorithm, i.e., the Augmented Lagrangian based Alternating Direction Inexact Newton method (ALADIN). Comprehensive evaluations across multiple system scenarios reveal a marginal performance slowdown compared to the centralized approach and the distributed approach executed on single machines -- a justified trade-off for enhanced data privacy. This investigation serves as empirical validation of the successful execution of distributed AC OPF within a geographically distributed environment, highlighting potential directions for future research. △ Less

Submitted 15 March, 2024; v1 submitted 1 February, 2024; originally announced February 2024.

Stable Linear Subspace Identification: A Machine Learning Approach

Authors: Loris Di Natale, Muhammad Zakwan, Bratislav Svetozarevic, Philipp Heer, Giancarlo Ferrari-Trecate, Colin N. Jones

Abstract: Machine Learning (ML) and linear System Identification (SI) have been historically developed independently. In this paper, we leverage well-established ML tools - especially the automatic differentiation framework - to introduce SIMBa, a family of discrete linear multi-step-ahead state-space SI methods using backpropagation. SIMBa relies on a novel Linear-Matrix-Inequality-based free parametrizati… ▽ More Machine Learning (ML) and linear System Identification (SI) have been historically developed independently. In this paper, we leverage well-established ML tools - especially the automatic differentiation framework - to introduce SIMBa, a family of discrete linear multi-step-ahead state-space SI methods using backpropagation. SIMBa relies on a novel Linear-Matrix-Inequality-based free parametrization of Schur matrices to ensure the stability of the identified model. We show how SIMBa generally outperforms traditional linear state-space SI methods, and sometimes significantly, although at the price of a higher computational burden. This performance gap is particularly remarkable compared to other SI methods with stability guarantees, where the gain is frequently above 25% in our investigations, hinting at SIMBa's ability to simultaneously achieve state-of-the-art fitting performance and enforce stability. Interestingly, these observations hold for a wide variety of input-output systems and on both simulated and real-world data, showcasing the flexibility of the proposed approach. We postulate that this new SI paradigm presents a great extension potential to identify structured nonlinear models from data, and we hence open-source SIMBa on https://github.com/Cemempamoi/simba. △ Less

Submitted 26 March, 2024; v1 submitted 6 November, 2023; originally announced November 2023.

Comments: Accepted at ECC 2024

Multi-Agent Bayesian Optimization with Coupled Black-Box and Affine Constraints

Authors: Wenjie Xu, Yuning Jiang, Bratislav Svetozarevic, Colin N. Jones

Abstract: This paper studies the problem of distributed multi-agent Bayesian optimization with both coupled black-box constraints and known affine constraints. A primal-dual distributed algorithm is proposed that achieves similar regret/violation bounds as those in the single-agent case for the black-box objective and constraint functions. Additionally, the algorithm guarantees an $\mathcal{O}(N\sqrt{T})$ b… ▽ More This paper studies the problem of distributed multi-agent Bayesian optimization with both coupled black-box constraints and known affine constraints. A primal-dual distributed algorithm is proposed that achieves similar regret/violation bounds as those in the single-agent case for the black-box objective and constraint functions. Additionally, the algorithm guarantees an $\mathcal{O}(N\sqrt{T})$ bound on the cumulative violation for the known affine constraints, where $N$ is the number of agents. Hence, it is ensured that the average of the samples satisfies the affine constraints up to the error $\mathcal{O}({N}/{\sqrt{T}})$. Furthermore, we characterize certain conditions under which our algorithm can bound a stronger metric of cumulative violation and provide best-iterate convergence without affine constraint. The method is then applied to both sampled instances from Gaussian processes and a real-world optimal power allocation problem for wireless communication; the results show that our method simultaneously provides close-to-optimal performance and maintains minor violations on average, corroborating our theoretical analysis. △ Less

Submitted 2 October, 2023; originally announced October 2023.

Data-driven adaptive building thermal controller tuning with constraints: A primal-dual contextual Bayesian optimization approach

Authors: Wenjie Xu, Bratislav Svetozarevic, Loris Di Natale, Philipp Heer, Colin N Jones

Abstract: We study the problem of tuning the parameters of a room temperature controller to minimize its energy consumption, subject to the constraint that the daily cumulative thermal discomfort of the occupants is below a given threshold. We formulate it as an online constrained black-box optimization problem where, on each day, we observe some relevant environmental context and adaptively select the cont… ▽ More We study the problem of tuning the parameters of a room temperature controller to minimize its energy consumption, subject to the constraint that the daily cumulative thermal discomfort of the occupants is below a given threshold. We formulate it as an online constrained black-box optimization problem where, on each day, we observe some relevant environmental context and adaptively select the controller parameters. In this paper, we propose to use a data-driven Primal-Dual Contextual Bayesian Optimization (PDCBO) approach to solve this problem. In a simulation case study on a single room, we apply our algorithm to tune the parameters of a Proportional Integral (PI) heating controller and the pre-heating time. Our results show that PDCBO can save up to 4.7% energy consumption compared to other state-of-the-art Bayesian optimization-based methods while keeping the daily thermal discomfort below the given tolerable threshold on average. Additionally, PDCBO can automatically track time-varying tolerable thresholds while existing methods fail to do so. We then study an alternative constrained tuning problem where we aim to minimize the thermal discomfort with a given energy budget. With this formulation, PDCBO reduces the average discomfort by up to 63% compared to state-of-the-art safe optimization methods while keeping the average daily energy consumption below the required threshold. △ Less

Submitted 1 October, 2023; originally announced October 2023.

Bayesian Optimization of Expensive Nested Grey-Box Functions

Authors: Wenjie Xu, Yuning Jiang, Bratislav Svetozarevic, Colin N. Jones

Abstract: We consider the problem of optimizing a grey-box objective function, i.e., nested function composed of both black-box and white-box functions. A general formulation for such grey-box problems is given, which covers the existing grey-box optimization formulations as special cases. We then design an optimism-driven algorithm to solve it. Under certain regularity assumptions, our algorithm achieves s… ▽ More We consider the problem of optimizing a grey-box objective function, i.e., nested function composed of both black-box and white-box functions. A general formulation for such grey-box problems is given, which covers the existing grey-box optimization formulations as special cases. We then design an optimism-driven algorithm to solve it. Under certain regularity assumptions, our algorithm achieves similar regret bound as that for the standard black-box Bayesian optimization algorithm, up to a constant multiplicative term depending on the Lipschitz constants of the functions considered. We further extend our method to the constrained case and discuss special cases. For the commonly used kernel functions, the regret bounds allow us to derive a convergence rate to the optimal solution. Experimental results show that our grey-box optimization method empirically improves the speed of finding the global optimal solution significantly, as compared to the standard black-box optimization algorithm. △ Less

Submitted 2 August, 2023; v1 submitted 8 June, 2023; originally announced June 2023.

Primal-Dual Contextual Bayesian Optimization for Control System Online Optimization with Time-Average Constraints

Authors: Wenjie Xu, Yuning Jiang, Bratislav Svetozarevic, Colin N. Jones

Abstract: This paper studies the problem of online performance optimization of constrained closed-loop control systems, where both the objective and the constraints are unknown black-box functions affected by exogenous time-varying contextual disturbances. A primal-dual contextual Bayesian optimization algorithm is proposed that achieves sublinear cumulative regret with respect to the dynamic optimal soluti… ▽ More This paper studies the problem of online performance optimization of constrained closed-loop control systems, where both the objective and the constraints are unknown black-box functions affected by exogenous time-varying contextual disturbances. A primal-dual contextual Bayesian optimization algorithm is proposed that achieves sublinear cumulative regret with respect to the dynamic optimal solution under certain regularity conditions. Furthermore, the algorithm achieves zero time-average constraint violation, ensuring that the average value of the constraint function satisfies the desired constraint. The method is applied to both sampled instances from Gaussian processes and a continuous stirred tank reactor parameter tuning problem; simulation results show that the method simultaneously provides close-to-optimal performance and maintains constraint feasibility on average. This contrasts current state-of-the-art methods, which either suffer from large cumulative regret or severe constraint violations for the case studies presented. △ Less

Submitted 20 September, 2023; v1 submitted 12 April, 2023; originally announced April 2023.

Understanding Frontline Workers' and Unhoused Individuals' Perspectives on AI Used in Homeless Services

Authors: Tzu-Sheng Kuo, Hong Shen, Jisoo Geum, Nev Jones, Jason I. Hong, Haiyi Zhu, Kenneth Holstein

Abstract: Recent years have seen growing adoption of AI-based decision-support systems (ADS) in homeless services, yet we know little about stakeholder desires and concerns surrounding their use. In this work, we aim to understand impacted stakeholders' perspectives on a deployed ADS that prioritizes scarce housing resources. We employed AI lifecycle comicboarding, an adapted version of the comicboarding me… ▽ More Recent years have seen growing adoption of AI-based decision-support systems (ADS) in homeless services, yet we know little about stakeholder desires and concerns surrounding their use. In this work, we aim to understand impacted stakeholders' perspectives on a deployed ADS that prioritizes scarce housing resources. We employed AI lifecycle comicboarding, an adapted version of the comicboarding method, to elicit stakeholder feedback and design ideas across various components of an AI system's design. We elicited feedback from county workers who operate the ADS daily, service providers whose work is directly impacted by the ADS, and unhoused individuals in the region. Our participants shared concerns and design suggestions around the AI system's overall objective, specific model design choices, dataset selection, and use in deployment. Our findings demonstrate that stakeholders, even without AI knowledge, can provide specific and critical feedback on an AI system's design and deployment, if empowered to do so. △ Less

Submitted 16 March, 2023; originally announced March 2023.

Journal ref: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23)

Violation-Aware Contextual Bayesian Optimization for Controller Performance Optimization with Unmodeled Constraints

Authors: Wenjie Xu, Colin N Jones, Bratislav Svetozarevic, Christopher R. Laughman, Ankush Chakrabarty

Abstract: We study the problem of performance optimization of closed-loop control systems with unmodeled dynamics. Bayesian optimization (BO) has been demonstrated to be effective for improving closed-loop performance by automatically tuning controller gains or reference setpoints in a model-free manner. However, BO methods have rarely been tested on dynamical systems with unmodeled constraints and time-var… ▽ More We study the problem of performance optimization of closed-loop control systems with unmodeled dynamics. Bayesian optimization (BO) has been demonstrated to be effective for improving closed-loop performance by automatically tuning controller gains or reference setpoints in a model-free manner. However, BO methods have rarely been tested on dynamical systems with unmodeled constraints and time-varying ambient conditions. In this paper, we propose a violation-aware contextual BO algorithm (VACBO) that optimizes closed-loop performance while simultaneously learning constraint-feasible solutions under time-varying ambient conditions. Unlike classical constrained BO methods which allow unlimited constraint violations, or 'safe' BO algorithms that are conservative and try to operate with near-zero violations, we allow budgeted constraint violations to improve constraint learning and accelerate optimization. We demonstrate the effectiveness of our proposed VACBO method for energy minimization of industrial vapor compression systems under time-varying ambient temperature and humidity. △ Less

Submitted 28 January, 2023; originally announced January 2023.

Comments: arXiv admin note: substantial text overlap with arXiv:2110.07479

Towards Scalable Physically Consistent Neural Networks: an Application to Data-driven Multi-zone Thermal Building Models

Authors: Loris Di Natale, Bratislav Svetozarevic, Philipp Heer, Colin Neil Jones

Abstract: With more and more data being collected, data-driven modeling methods have been gaining in popularity in recent years. While physically sound, classical gray-box models are often cumbersome to identify and scale, and their accuracy might be hindered by their limited expressiveness. On the other hand, classical black-box methods, typically relying on Neural Networks (NNs) nowadays, often achieve im… ▽ More With more and more data being collected, data-driven modeling methods have been gaining in popularity in recent years. While physically sound, classical gray-box models are often cumbersome to identify and scale, and their accuracy might be hindered by their limited expressiveness. On the other hand, classical black-box methods, typically relying on Neural Networks (NNs) nowadays, often achieve impressive performance, even at scale, by deriving statistical patterns from data. However, they remain completely oblivious to the underlying physical laws, which may lead to potentially catastrophic failures if decisions for real-world physical systems are based on them. Physically Consistent Neural Networks (PCNNs) were recently developed to address these aforementioned issues, ensuring physical consistency while still leveraging NNs to attain state-of-the-art accuracy. In this work, we scale PCNNs to model building temperature dynamics and propose a thorough comparison with classical gray-box and black-box methods. More precisely, we design three distinct PCNN extensions, thereby exemplifying the modularity and flexibility of the architecture, and formally prove their physical consistency. In the presented case study, PCNNs are shown to achieve state-of-the-art accuracy, even outperforming classical NN-based models despite their constrained structure. Our investigations furthermore provide a clear illustration of NNs achieving seemingly good performance while remaining completely physics-agnostic, which can be misleading in practice. While this performance comes at the cost of computational complexity, PCNNs on the other hand show accuracy improvements of 17-35% compared to all other physically consistent methods, paving the way for scalable physically consistent models with state-of-the-art performance. △ Less

Submitted 4 April, 2023; v1 submitted 23 December, 2022; originally announced December 2022.

Comments: Accepted in Applied Energy

Minimizing Age of Information in Spatially Distributed Random Access Wireless Networks

Authors: Nicholas Jones, Eytan Modiano

Abstract: We analyze Age of Information (AoI) in wireless networks where nodes use a spatially adaptive random access scheme to send status updates to a central base station. We show that the set of achievable AoI in this setting is convex, and design policies to minimize weighted sum, min-max, and proportionally fair AoI by setting transmission probabilities as a function of node locations. We show that un… ▽ More We analyze Age of Information (AoI) in wireless networks where nodes use a spatially adaptive random access scheme to send status updates to a central base station. We show that the set of achievable AoI in this setting is convex, and design policies to minimize weighted sum, min-max, and proportionally fair AoI by setting transmission probabilities as a function of node locations. We show that under the capture model, when the spatial topology of the network is considered, AoI can be significantly improved, and we obtain tight performance bounds on weighted sum and min-max AoI. Finally, we design a policy where each node sets its transmission probability based only on its own distance from the base station, when it does not know the positions of other nodes, and show that it converges to the optimal proportionally fair policy as the size of the network goes to infinity. △ Less

Submitted 4 January, 2023; v1 submitted 7 December, 2022; originally announced December 2022.

Comments: Accepted to IEEE INFOCOM 2023

Fresh-CSMA: A Distributed Protocol for Minimizing Age of Information

Authors: Vishrant Tripathi, Nicholas Jones, Eytan Modiano

Abstract: We consider the design of distributed scheduling algorithms that minimize age of information in single-hop wireless networks. The centralized max-weight policy is known to be nearly optimal in this setting; hence, our goal is to design a distributed CSMA scheme that can mimic its performance. To that end, we propose a distributed protocol called Fresh-CSMA and show that in an idealized setting, Fr… ▽ More We consider the design of distributed scheduling algorithms that minimize age of information in single-hop wireless networks. The centralized max-weight policy is known to be nearly optimal in this setting; hence, our goal is to design a distributed CSMA scheme that can mimic its performance. To that end, we propose a distributed protocol called Fresh-CSMA and show that in an idealized setting, Fresh-CSMA can match the scheduling decisions of the max-weight policy with high probability in each frame, and also match the theoretical performance guarantees of the max-weight policy over the entire time horizon. We then consider a more realistic setting and study the impact of protocol parameters on the probability of collisions and the overhead caused by the distributed nature of the protocol. We also consider the monitoring of Markov sources and extend our approach to CSMA protocols that incorporate Age of Incorrect Information (AoII) instead of AoI. Finally, we provide simulations that support our theoretical results and show that the performance gap between the ideal and realistic versions of Fresh-CSMA is small. △ Less

Submitted 12 July, 2023; v1 submitted 6 December, 2022; originally announced December 2022.

Comments: Accepted at IEEE INFOCOM 2023

Computationally Efficient Reinforcement Learning: Targeted Exploration leveraging Simple Rules

Authors: Loris Di Natale, Bratislav Svetozarevic, Philipp Heer, Colin N. Jones

Abstract: Model-free Reinforcement Learning (RL) generally suffers from poor sample complexity, mostly due to the need to exhaustively explore the state-action space to find well-performing policies. On the other hand, we postulate that expert knowledge of the system often allows us to design simple rules we expect good policies to follow at all times. In this work, we hence propose a simple yet effective m… ▽ More Model-free Reinforcement Learning (RL) generally suffers from poor sample complexity, mostly due to the need to exhaustively explore the state-action space to find well-performing policies. On the other hand, we postulate that expert knowledge of the system often allows us to design simple rules we expect good policies to follow at all times. In this work, we hence propose a simple yet effective modification of continuous actor-critic frameworks to incorporate such rules and avoid regions of the state-action space that are known to be suboptimal, thereby significantly accelerating the convergence of RL agents. Concretely, we saturate the actions chosen by the agent if they do not comply with our intuition and, critically, modify the gradient update step of the policy to ensure the learning process is not affected by the saturation step. On a room temperature control case study, it allows agents to converge to well-performing policies up to 6-7x faster than classical agents without computational overhead and while retaining good final performance. △ Less

Submitted 12 September, 2023; v1 submitted 29 November, 2022; originally announced November 2022.

Comments: Accepted to CDC 2023

CONFIG: Constrained Efficient Global Optimization for Closed-Loop Control System Optimization with Unmodeled Constraints

Authors: Wenjie Xu, Yuning Jiang, Bratislav Svetozarevic, Colin N. Jones

Abstract: In this paper, the CONFIG algorithm, a simple and provably efficient constrained global optimization algorithm, is applied to optimize the closed-loop control performance of an unknown system with unmodeled constraints. Existing Gaussian process based closed-loop optimization methods, either can only guarantee local convergence (e.g., SafeOPT), or have no known optimality guarantee (e.g., constrai… ▽ More In this paper, the CONFIG algorithm, a simple and provably efficient constrained global optimization algorithm, is applied to optimize the closed-loop control performance of an unknown system with unmodeled constraints. Existing Gaussian process based closed-loop optimization methods, either can only guarantee local convergence (e.g., SafeOPT), or have no known optimality guarantee (e.g., constrained expected improvement) at all, whereas the recently introduced CONFIG algorithm has been proven to enjoy a theoretical global optimality guarantee. In this study, we demonstrate the effectiveness of CONFIG algorithm in the applications. The algorithm is first applied to an artificial numerical benchmark problem to corroborate its effectiveness. It is then applied to a classical constrained steady-state optimization problem of a continuous stirred-tank reactor. Simulation results show that our CONFIG algorithm can achieve performance competitive with the popular CEI (Constrained Expected Improvement) algorithm, which has no known optimality guarantee. As such, the CONFIG algorithm offers a new tool, with both a provable global optimality guarantee and competitive empirical performance, to optimize the closed-loop control performance for a system with soft unmodeled constraints. Last, but not least, the open-source code is available as a python package to facilitate future applications. △ Less

Submitted 18 December, 2022; v1 submitted 21 November, 2022; originally announced November 2022.

Physically Consistent Neural ODEs for Learning Multi-Physics Systems

Authors: Muhammad Zakwan, Loris Di Natale, Bratislav Svetozarevic, Philipp Heer, Colin N. Jones, Giancarlo Ferrari Trecate

Abstract: Despite the immense success of neural networks in modeling system dynamics from data, they often remain physics-agnostic black boxes. In the particular case of physical systems, they might consequently make physically inconsistent predictions, which makes them unreliable in practice. In this paper, we leverage the framework of Irreversible port-Hamiltonian Systems (IPHS), which can describe most m… ▽ More Despite the immense success of neural networks in modeling system dynamics from data, they often remain physics-agnostic black boxes. In the particular case of physical systems, they might consequently make physically inconsistent predictions, which makes them unreliable in practice. In this paper, we leverage the framework of Irreversible port-Hamiltonian Systems (IPHS), which can describe most multi-physics systems, and rely on Neural Ordinary Differential Equations (NODEs) to learn their parameters from data. Since IPHS models are consistent with the first and second principles of thermodynamics by design, so are the proposed Physically Consistent NODEs (PC-NODEs). Furthermore, the NODE training procedure allows us to seamlessly incorporate prior knowledge of the system properties in the learned dynamics. We demonstrate the effectiveness of the proposed method by learning the thermodynamics of a building from the real-world measurements and the dynamics of a simulated gas-piston system. Thanks to the modularity and flexibility of the IPHS framework, PC-NODEs can be extended to learn physically consistent models of multi-physics distributed systems. △ Less

Submitted 11 November, 2022; originally announced November 2022.

Comments: First two authors contributed equally. Submitted to IFAC 2023

Lower Bounds on the Worst-Case Complexity of Efficient Global Optimization

Authors: Wenjie Xu, Yuning Jiang, Emilio T. Maddalena, Colin N. Jones

Abstract: Efficient global optimization is a widely used method for optimizing expensive black-box functions such as tuning hyperparameter, and designing new material, etc. Despite its popularity, less attention has been paid to analyzing the inherent hardness of the problem although, given its extensive use, it is important to understand the fundamental limits of efficient global optimization algorithms. I… ▽ More Efficient global optimization is a widely used method for optimizing expensive black-box functions such as tuning hyperparameter, and designing new material, etc. Despite its popularity, less attention has been paid to analyzing the inherent hardness of the problem although, given its extensive use, it is important to understand the fundamental limits of efficient global optimization algorithms. In this paper, we study the worst-case complexity of the efficient global optimization problem and, in contrast to existing kernel-specific results, we derive a unified lower bound for the complexity of efficient global optimization in terms of the metric entropy of a ball in its corresponding reproducing kernel Hilbert space~(RKHS). Specifically, we show that if there exists a deterministic algorithm that achieves suboptimality gap smaller than $ε$ for any function $f\in S$ in $T$ function evaluations, it is necessary that $T$ is at least $Ω\left(\frac{\log\mathcal{N}(S(\mathcal{X}), 4ε,\|\cdot\|_\infty)}{\log(\frac{R}ε)}\right)$, where $\mathcal{N}(\cdot,\cdot,\cdot)$ is the covering number, $S$ is the ball centered at $0$ with radius $R$ in the RKHS and $S(\mathcal{X})$ is the restriction of $S$ over the feasible set $\mathcal{X}$. Moreover, we show that this lower bound nearly matches the upper bound attained by non-adaptive search algorithms for the commonly used squared exponential kernel and the Matérn kernel with a large smoothness parameter $ν$, up to a replacement of $d/2$ by $d$ and a logarithmic term $\log\frac{R}ε$. That is to say, our lower bound is nearly optimal for these kernels. △ Less

Submitted 20 September, 2022; originally announced September 2022.

Stability Verification of Neural Network Controllers using Mixed-Integer Programming

Authors: Roland Schwan, Colin N. Jones, Daniel Kuhn

Abstract: We propose a framework for the stability verification of Mixed-Integer Linear Programming (MILP) representable control policies. This framework compares a fixed candidate policy, which admits an efficient parameterization and can be evaluated at a low computational cost, against a fixed baseline policy, which is known to be stable but expensive to evaluate. We provide sufficient conditions for the… ▽ More We propose a framework for the stability verification of Mixed-Integer Linear Programming (MILP) representable control policies. This framework compares a fixed candidate policy, which admits an efficient parameterization and can be evaluated at a low computational cost, against a fixed baseline policy, which is known to be stable but expensive to evaluate. We provide sufficient conditions for the closed-loop stability of the candidate policy in terms of the worst-case approximation error with respect to the baseline policy, and we show that these conditions can be checked by solving a Mixed-Integer Quadratic Program (MIQP). Additionally, we demonstrate that an outer and inner approximation of the stability region of the candidate policy can be computed by solving an MILP. The proposed framework is sufficiently general to accommodate a broad range of candidate policies including ReLU Neural Networks (NNs), optimal solution maps of parametric quadratic programs, and Model Predictive Control (MPC) policies. We also present an open-source toolbox in Python based on the proposed framework, which allows for the easy verification of custom NN architectures and MPC formulations. We showcase the flexibility and reliability of our framework in the context of a DC-DC power converter case study and investigate its computational complexity. △ Less

Submitted 31 May, 2023; v1 submitted 27 June, 2022; originally announced June 2022.

Lessons Learned from Data-Driven Building Control Experiments: Contrasting Gaussian Process-based MPC, Bilevel DeePC, and Deep Reinforcement Learning

Authors: Loris Di Natale, Yingzhao Lian, Emilio T. Maddalena, Jicheng Shi, Colin N. Jones

Abstract: This manuscript offers the perspective of experimentalists on a number of modern data-driven techniques: model predictive control relying on Gaussian processes, adaptive data-driven control based on behavioral theory, and deep reinforcement learning. These techniques are compared in terms of data requirements, ease of use, computational burden, and robustness in the context of real-world applicati… ▽ More This manuscript offers the perspective of experimentalists on a number of modern data-driven techniques: model predictive control relying on Gaussian processes, adaptive data-driven control based on behavioral theory, and deep reinforcement learning. These techniques are compared in terms of data requirements, ease of use, computational burden, and robustness in the context of real-world applications. Our remarks and observations stem from a number of experimental investigations carried out in the field of building control in diverse environments, from lecture halls and apartment spaces to a hospital surgery center. The final goal is to support others in identifying what technique is best suited to tackle their own problems. △ Less

Submitted 31 May, 2022; originally announced May 2022.

Over-the-Air Federated Learning via Second-Order Optimization

Authors: Peng Yang, Yuning Jiang, Ting Wang, Yong Zhou, Yuanming Shi, Colin N. Jones

Abstract: Federated learning (FL) is a promising learning paradigm that can tackle the increasingly prominent isolated data islands problem while keeping users' data locally with privacy and security guarantees. However, FL could result in task-oriented data traffic flows over wireless networks with limited radio resources. To design communication-efficient FL, most of the existing studies employ the first-… ▽ More Federated learning (FL) is a promising learning paradigm that can tackle the increasingly prominent isolated data islands problem while keeping users' data locally with privacy and security guarantees. However, FL could result in task-oriented data traffic flows over wireless networks with limited radio resources. To design communication-efficient FL, most of the existing studies employ the first-order federated optimization approach that has a slow convergence rate. This however results in excessive communication rounds for local model updates between the edge devices and edge server. To address this issue, in this paper, we instead propose a novel over-the-air second-order federated optimization algorithm to simultaneously reduce the communication rounds and enable low-latency global model aggregation. This is achieved by exploiting the waveform superposition property of a multi-access channel to implement the distributed second-order optimization algorithm over wireless networks. The convergence behavior of the proposed algorithm is further characterized, which reveals a linear-quadratic convergence rate with an accumulative error term in each iteration. We thus propose a system optimization approach to minimize the accumulated error gap by joint device selection and beamforming design. Numerical results demonstrate the system and communication efficiency compared with the state-of-the-art approaches. △ Less

Submitted 29 March, 2022; originally announced March 2022.

Comments: 30 pages, 9 figures

Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control

Authors: Loris Di Natale, Bratislav Svetozarevic, Philipp Heer, Colin N. Jones

Abstract: Replacing poorly performing existing controllers with smarter solutions will decrease the energy intensity of the building sector. Recently, controllers based on Deep Reinforcement Learning (DRL) have been shown to be more effective than conventional baselines. However, since the optimal solution is usually unknown, it is still unclear if DRL agents are attaining near-optimal performance in genera… ▽ More Replacing poorly performing existing controllers with smarter solutions will decrease the energy intensity of the building sector. Recently, controllers based on Deep Reinforcement Learning (DRL) have been shown to be more effective than conventional baselines. However, since the optimal solution is usually unknown, it is still unclear if DRL agents are attaining near-optimal performance in general or if there is still a large gap to bridge. In this paper, we investigate the performance of DRL agents compared to the theoretically optimal solution. To that end, we leverage Physically Consistent Neural Networks (PCNNs) as simulation environments, for which optimal control inputs are easy to compute. Furthermore, PCNNs solely rely on data to be trained, avoiding the difficult physics-based modeling phase, while retaining physical consistency. Our results hint that DRL agents not only clearly outperform conventional rule-based controllers, they furthermore attain near-optimal performance. △ Less

Submitted 10 March, 2022; originally announced March 2022.

Comments: Submitted to IEEE ICCA 2022 - 6 pages, 5 figures

Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learning

Authors: Wenzhi Fang, Ziyi Yu, Yuning Jiang, Yuanming Shi, Colin N. Jones, Yong Zhou

Abstract: Federated learning (FL), as an emerging edge artificial intelligence paradigm, enables many edge devices to collaboratively train a global model without sharing their private data. To enhance the training efficiency of FL, various algorithms have been proposed, ranging from first-order to second-order methods. However, these algorithms cannot be applied in scenarios where the gradient information… ▽ More Federated learning (FL), as an emerging edge artificial intelligence paradigm, enables many edge devices to collaboratively train a global model without sharing their private data. To enhance the training efficiency of FL, various algorithms have been proposed, ranging from first-order to second-order methods. However, these algorithms cannot be applied in scenarios where the gradient information is not available, e.g., federated black-box attack and federated hyperparameter tuning. To address this issue, in this paper we propose a derivative-free federated zeroth-order optimization (FedZO) algorithm featured by performing multiple local updates based on stochastic gradient estimators in each communication round and enabling partial device participation. Under non-convex settings, we derive the convergence performance of the FedZO algorithm on non-independent and identically distributed data and characterize the impact of the numbers of local iterates and participating edge devices on the convergence. To enable communication-efficient FedZO over wireless networks, we further propose an over-the-air computation (AirComp) assisted FedZO algorithm. With an appropriate transceiver design, we show that the convergence of AirComp-assisted FedZO can still be preserved under certain signal-to-noise ratio conditions. Simulation results demonstrate the effectiveness of the FedZO algorithm and validate the theoretical observations. △ Less

Submitted 10 October, 2022; v1 submitted 24 January, 2022; originally announced January 2022.

Comments: This work was accepted to Transaction on Signal Processing

Physically Consistent Neural Networks for building thermal modeling: theory and analysis

Authors: Loris Di Natale, Bratislav Svetozarevic, Philipp Heer, Colin N. Jones

Abstract: Due to their high energy intensity, buildings play a major role in the current worldwide energy transition. Building models are ubiquitous since they are needed at each stage of the life of buildings, i.e. for design, retrofitting, and control operations. Classical white-box models, based on physical equations, are bound to follow the laws of physics but the specific design of their underlying str… ▽ More Due to their high energy intensity, buildings play a major role in the current worldwide energy transition. Building models are ubiquitous since they are needed at each stage of the life of buildings, i.e. for design, retrofitting, and control operations. Classical white-box models, based on physical equations, are bound to follow the laws of physics but the specific design of their underlying structure might hinder their expressiveness and hence their accuracy. On the other hand, black-box models are better suited to capture nonlinear building dynamics and thus can often achieve better accuracy, but they require a lot of data and might not follow the laws of physics, a problem that is particularly common for neural network (NN) models. To counter this known generalization issue, physics-informed NNs have recently been introduced, where researchers introduce prior knowledge in the structure of NNs to ground them in known underlying physical laws and avoid classical NN generalization issues. In this work, we present a novel physics-informed NN architecture, dubbed Physically Consistent NN (PCNN), which only requires past operational data and no engineering overhead, including prior knowledge in a linear module running in parallel to a classical NN. We formally prove that such networks are physically consistent - by design and even on unseen data - with respect to different control inputs and temperatures outside and in neighboring zones. We demonstrate their performance on a case study, where the PCNN attains an accuracy up to 40% better than a classical physics-based resistance-capacitance model on 3-day long prediction horizons. Furthermore, despite their constrained structure, PCNNs attain similar performance to classical NNs on the validation data, overfitting the training data less and retaining high expressiveness to tackle the generalization issue. △ Less

Submitted 11 July, 2022; v1 submitted 6 December, 2021; originally announced December 2021.

Comments: Preprint submitted to Applied Energy. 13 pages in the main text + 5 in appendix, 11 figures

Geodesic statistics for random network families

Authors: Sahil Loomba, Nick S. Jones

Abstract: A key task in the study of networked systems is to derive local and global properties that impact connectivity, synchronizability, and robustness. Computing shortest paths or geodesics in the network yields measures of node centrality and network connectivity that can contribute to explain such phenomena. We derive an analytic distribution of shortest path lengths, on the giant component in the su… ▽ More A key task in the study of networked systems is to derive local and global properties that impact connectivity, synchronizability, and robustness. Computing shortest paths or geodesics in the network yields measures of node centrality and network connectivity that can contribute to explain such phenomena. We derive an analytic distribution of shortest path lengths, on the giant component in the supercritical regime or on small components in the subcritical regime, of any sparse (possibly directed) graph with conditionally independent edges, in the infinite-size limit. We provide specific results for widely used network families like stochastic block models, dot-product graphs, random geometric graphs, and graphons. The survival function of the shortest path length distribution possesses a simple closed-form lower bound which is asymptotically tight for finite lengths, has a natural interpretation of traversing independent geodesics in the network, and delivers novel insight in the above network families. Notably, the shortest path length distribution allows us to derive, for the network families above, important graph properties like the bond percolation threshold, size of the giant component, average shortest path length, and closeness and betweenness centralities. We also provide a corroborative analysis of a set of 20 empirical networks. This unifying framework demonstrates how geodesic statistics for a rich family of random graphs can be computed cheaply without having access to true or simulated networks, especially when they are sparse but prohibitively large. △ Less

Submitted 3 November, 2021; originally announced November 2021.

Comments: 32 pages, 12 figures

VABO: Violation-Aware Bayesian Optimization for Closed-Loop Control Performance Optimization with Unmodeled Constraints

Authors: Wenjie Xu, Colin N Jones, Bratislav Svetozarevic, Christopher R. Laughman, Ankush Chakrabarty

Abstract: We study the problem of performance optimization of closed-loop control systems with unmodeled dynamics. Bayesian optimization (BO) has been demonstrated effective for improving closed-loop performance by automatically tuning controller gains or reference setpoints in a model-free manner. However, BO methods have rarely been tested on dynamical systems with unmodeled constraints. In this paper, we… ▽ More We study the problem of performance optimization of closed-loop control systems with unmodeled dynamics. Bayesian optimization (BO) has been demonstrated effective for improving closed-loop performance by automatically tuning controller gains or reference setpoints in a model-free manner. However, BO methods have rarely been tested on dynamical systems with unmodeled constraints. In this paper, we propose a violation-aware BO algorithm (VABO) that optimizes closed-loop performance while simultaneously learning constraint-feasible solutions. Unlike classical constrained BO methods which allow an unlimited constraint violations, or safe BO algorithms that are conservative and try to operate with near-zero violations, we allow budgeted constraint violations to improve constraint learning and accelerate optimization. We demonstrate the effectiveness of our proposed VABO method for energy minimization of industrial vapor compression systems. △ Less

Submitted 14 October, 2021; originally announced October 2021.

Mobilkit: A Python Toolkit for Urban Resilience and Disaster Risk Management Analytics using High Frequency Human Mobility Data

Authors: Enrico Ubaldi, Takahiro Yabe, Nicholas K. W. Jones, Maham Faisal Khan, Satish V. Ukkusuri, Riccardo Di Clemente, Emanuele Strano

Abstract: Increasingly available high-frequency location datasets derived from smartphones provide unprecedented insight into trajectories of human mobility. These datasets can play a significant and growing role in informing preparedness and response to natural disasters. However, limited tools exist to enable rapid analytics using mobility data, and tend not to be tailored specifically for disaster risk m… ▽ More Increasingly available high-frequency location datasets derived from smartphones provide unprecedented insight into trajectories of human mobility. These datasets can play a significant and growing role in informing preparedness and response to natural disasters. However, limited tools exist to enable rapid analytics using mobility data, and tend not to be tailored specifically for disaster risk management. We present an open-source, Python-based toolkit designed to conduct replicable and scalable post-disaster analytics using GPS location data. Privacy, system capabilities, and potential expansions of \textit{Mobilkit} are discussed. △ Less

Submitted 16 September, 2021; v1 submitted 29 July, 2021; originally announced July 2021.

Comments: 3 pages, 1 figure, KDD KDD Workshop on Data-driven Humanitarian Mapping, 27th ACM SIGKDD Conference

ACM Class: J.2

Journal ref: Journal of Open Source Software, 9(95), 5201, 2024

Robust Uncertainty Bounds in Reproducing Kernel Hilbert Spaces: A Convex Optimization Approach

Authors: Paul Scharnhorst, Emilio T. Maddalena, Yuning Jiang, Colin N. Jones

Abstract: The problem of establishing out-of-sample bounds for the values of an unkonwn ground-truth function is considered. Kernels and their associated Hilbert spaces are the main formalism employed herein along with an observational model where outputs are corrupted by bounded measurement noise. The noise can originate from any compactly supported distribution and no independence assumptions are made on… ▽ More The problem of establishing out-of-sample bounds for the values of an unkonwn ground-truth function is considered. Kernels and their associated Hilbert spaces are the main formalism employed herein along with an observational model where outputs are corrupted by bounded measurement noise. The noise can originate from any compactly supported distribution and no independence assumptions are made on the available data. In this setting, we show how computing tight, finite-sample uncertainty bounds amounts to solving parametric quadratically constrained linear programs. Next, properties of our approach are established and its relationship with another methods is studied. Numerical experiments are presented to exemplify how the theory can be applied in a number of scenarios, and to contrast it with other closed-form alternatives. △ Less

Submitted 12 September, 2022; v1 submitted 19 April, 2021; originally announced April 2021.

Comments: 13 pages, 7 figures

Modularity maximisation for graphons

Authors: Florian Klimm, Nick S. Jones, Michael T. Schaub

Abstract: Networks are a widely-used tool to investigate the large-scale connectivity structure in complex systems and graphons have been proposed as an infinite size limit of dense networks. The detection of communities or other meso-scale structures is a prominent topic in network science as it allows the identification of functional building blocks in complex systems. When such building blocks may be pre… ▽ More Networks are a widely-used tool to investigate the large-scale connectivity structure in complex systems and graphons have been proposed as an infinite size limit of dense networks. The detection of communities or other meso-scale structures is a prominent topic in network science as it allows the identification of functional building blocks in complex systems. When such building blocks may be present in graphons is an open question. In this paper, we define a graphon-modularity and demonstrate that it can be maximised to detect communities in graphons. We then investigate specific synthetic graphons and show that they may show a wide range of different community structures. We also reformulate the graphon-modularity maximisation as a continuous optimisation problem and so prove the optimal community structure or lack thereof for some graphons, something that is usually not possible for networks. Furthermore, we demonstrate that estimating a graphon from network data as an intermediate step can improve the detection of communities, in comparison with exclusively maximising the modularity of the network. While the choice of graphon-estimator may strongly influence the accord between the community structure of a network and its estimated graphon, we find that there is a substantial overlap if an appropriate estimator is used. Our study demonstrates that community detection for graphons is possible and may serve as a privacy-preserving way to cluster network data. △ Less

Submitted 2 January, 2021; originally announced January 2021.

Influencing dynamics on social networks without knowledge of network microstructure

Authors: Matthew Garrod, Nick S. Jones

Abstract: Social network based information campaigns can be used for promoting beneficial health behaviours and mitigating polarisation (e.g. regarding climate change or vaccines). Network-based intervention strategies typically rely on full knowledge of network structure. It is largely not possible or desirable to obtain population-level social network data due to availability and privacy issues. It is eas… ▽ More Social network based information campaigns can be used for promoting beneficial health behaviours and mitigating polarisation (e.g. regarding climate change or vaccines). Network-based intervention strategies typically rely on full knowledge of network structure. It is largely not possible or desirable to obtain population-level social network data due to availability and privacy issues. It is easier to obtain information about individuals' attributes (e.g. age, income), which are jointly informative of an individual's opinions and their social network position. We investigate strategies for influencing the system state in a statistical mechanics based model of opinion formation. Using synthetic and data based examples we illustrate the advantages of implementing coarse-grained influence strategies on Ising models with modular structure in the presence of external fields. Our work provides a scalable methodology for influencing Ising systems on large graphs and the first exploration of the Ising influence problem in the presence of ambient (social) fields. By exploiting the observation that strong ambient fields can simplify control of networked dynamics, our findings open the possibility of efficiently computing and implementing public information campaigns using insights from social network theory without costly or invasive levels of data collection. △ Less

Submitted 27 July, 2021; v1 submitted 11 November, 2020; originally announced November 2020.

Human-centric Dialog Training via Offline Reinforcement Learning

Authors: Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Shane Gu, Rosalind Picard

Abstract: How can we train a dialog model to produce better conversations by learning from human feedback, without the risk of humans teaching it harmful chat behaviors? We start by hosting models online, and gather human feedback from real-time, open-ended conversations, which we then use to train and improve the models using offline reinforcement learning (RL). We identify implicit conversational cues inc… ▽ More How can we train a dialog model to produce better conversations by learning from human feedback, without the risk of humans teaching it harmful chat behaviors? We start by hosting models online, and gather human feedback from real-time, open-ended conversations, which we then use to train and improve the models using offline reinforcement learning (RL). We identify implicit conversational cues including language similarity, elicitation of laughter, sentiment, and more, which indicate positive human feedback, and embed these in multiple reward functions. A well-known challenge is that learning an RL policy in an offline setting usually fails due to the lack of ability to explore and the tendency to make over-optimistic estimates of future reward. These problems become even harder when using RL for language models, which can easily have a 20,000 action vocabulary and many possible reward functions. We solve the challenge by developing a novel class of offline RL algorithms. These algorithms use KL-control to penalize divergence from a pre-trained prior language model, and use a new strategy to make the algorithm pessimistic, instead of optimistic, in the face of uncertainty. We test the resulting dialog model with ratings from 80 users in an open-domain setting and find it achieves significant improvements over existing deep offline RL approaches. The novel offline RL method is viable for improving any existing generative dialog model using a static dataset of human feedback. △ Less

Submitted 12 October, 2020; originally announced October 2020.

Comments: To appear in EMNLP 2020 (long paper)

Inference of a universal social scale and segregation measures using social connectivity kernels

Authors: Till Hoffmann, Nick S. Jones

Abstract: How people connect with one another is a fundamental question in the social sciences, and the resulting social networks can have a profound impact on our daily lives. Blau offered a powerful explanation: people connect with one another based on their positions in a social space. Yet a principled measure of social distance, allowing comparison within and between societies, remains elusive. We use t… ▽ More How people connect with one another is a fundamental question in the social sciences, and the resulting social networks can have a profound impact on our daily lives. Blau offered a powerful explanation: people connect with one another based on their positions in a social space. Yet a principled measure of social distance, allowing comparison within and between societies, remains elusive. We use the connectivity kernel of conditionally-independent edge models to develop a family of segregation statistics with desirable properties: they offer an intuitive and universal characteristic scale on social space (facilitating comparison across datasets and societies), are applicable to multivariate and mixed node attributes, and capture segregation at the level of individuals, pairs of individuals, and society as a whole. We show that the segregation statistics can induce a metric on Blau space (a space spanned by the attributes of the members of society) and provide maps of two societies. Under a Bayesian paradigm, we infer the parameters of the connectivity kernel from eleven ego-network datasets collected in four surveys in the United Kingdom and United States. The importance of different dimensions of Blau space is similar across time and location, suggesting a macroscopically stable social fabric. Physical separation and age differences have the most significant impact on segregation within friendship networks with implications for intergenerational mixing and isolation in later stages of life. △ Less

Submitted 28 October, 2020; v1 submitted 12 August, 2020; originally announced August 2020.

Comments: Article: 23 pages, 3 figures. Supplementary material: 8 pages, 1 figure

Journal ref: J. R. Soc. Interface. 17: 20200638 (2020)

Deterministic error bounds for kernel-based learning techniques under bounded noise

Authors: Emilio T. Maddalena, Paul Scharnhorst, Colin N. Jones

Abstract: We consider the problem of reconstructing a function from a finite set of noise-corrupted samples. Two kernel algorithms are analyzed, namely kernel ridge regression and $\varepsilon$-support vector regression. By assuming the ground-truth function belongs to the reproducing kernel Hilbert space of the chosen kernel, and the measurement noise affecting the dataset is bounded, we adopt an approxima… ▽ More We consider the problem of reconstructing a function from a finite set of noise-corrupted samples. Two kernel algorithms are analyzed, namely kernel ridge regression and $\varepsilon$-support vector regression. By assuming the ground-truth function belongs to the reproducing kernel Hilbert space of the chosen kernel, and the measurement noise affecting the dataset is bounded, we adopt an approximation theory viewpoint to establish \textit{deterministic}, finite-sample error bounds for the two models. Finally, we discuss their connection with Gaussian processes and two numerical examples are provided. In establishing our inequalities, we hope to help bring the fields of non-parametric kernel learning and system identification for robust control closer to each other. △ Less

Submitted 31 July, 2021; v1 submitted 10 August, 2020; originally announced August 2020.

Comments: 18 pages, 2 figures

Cons-free Programs and Complexity Classes between LOGSPACE and PTIME

Authors: Neil D. Jones, Siddharth Bhaskar, Cynthia Kop, Jakob Grue Simonsen

Abstract: Programming language concepts are used to give some new perspectives on a long-standing open problem: is logspace = ptime ? Programming language concepts are used to give some new perspectives on a long-standing open problem: is logspace = ptime ? △ Less

Submitted 6 August, 2020; originally announced August 2020.

Comments: In Proceedings VPT/HCVS 2020, arXiv:2008.02483

Journal ref: EPTCS 320, 2020, pp. 65-79

Uncovering socioeconomic gaps in mobility reduction during the COVID-19 pandemic using location data

Authors: Samuel P. Fraiberger, Pablo Astudillo, Lorenzo Candeago, Alex Chunet, Nicholas K. W. Jones, Maham Faisal Khan, Bruno Lepri, Nancy Lozano Gracia, Lorenzo Lucchini, Emanuele Massaro, Aleister Montfort

Abstract: Using smartphone location data from Colombia, Mexico, and Indonesia, we investigate how non-pharmaceutical policy interventions intended to mitigate the spread of the COVID-19 pandemic impact human mobility. In all three countries, we find that following the implementation of mobility restriction measures, human movement decreased substantially. Importantly, we also uncover large and persistent di… ▽ More Using smartphone location data from Colombia, Mexico, and Indonesia, we investigate how non-pharmaceutical policy interventions intended to mitigate the spread of the COVID-19 pandemic impact human mobility. In all three countries, we find that following the implementation of mobility restriction measures, human movement decreased substantially. Importantly, we also uncover large and persistent differences in mobility reduction between wealth groups: on average, users in the top decile of wealth reduced their mobility up to twice as much as users in the bottom decile. For decision-makers seeking to efficiently allocate resources to response efforts, these findings highlight that smartphone location data can be leveraged to tailor policies to the needs of specific socioeconomic groups, especially the most vulnerable. △ Less

Submitted 27 July, 2020; v1 submitted 26 June, 2020; originally announced June 2020.

Comments: to appear in KDD 2020, Workshop on Humanitarian Mapping

Reconstructing Maps from Text

Authors: Johnathan E. Avery, Robert L. Goldstone, Michael N. Jones

Abstract: Previous research has demonstrated that Distributional Semantic Models (DSMs) are capable of reconstructing maps from news corpora (Louwerse & Zwaan, 2009) and novels (Louwerse & Benesh, 2012). The capacity for reproducing maps is surprising since DSMs notoriously lack perceptual grounding (De Vega et al., 2012). In this paper we investigate the statistical sources required in language to infer ma… ▽ More Previous research has demonstrated that Distributional Semantic Models (DSMs) are capable of reconstructing maps from news corpora (Louwerse & Zwaan, 2009) and novels (Louwerse & Benesh, 2012). The capacity for reproducing maps is surprising since DSMs notoriously lack perceptual grounding (De Vega et al., 2012). In this paper we investigate the statistical sources required in language to infer maps, and resulting constraints placed on mechanisms of semantic representation. Study 1 brings word co-occurrence under experimental control to demonstrate that direct co-occurrence in language is necessary for traditional DSMs to successfully reproduce maps. Study 2 presents an instance-based DSM that is capable of reconstructing maps independent of the frequency of co-occurrence of city names. △ Less

Submitted 18 May, 2020; originally announced May 2020.

Inference and Influence of Large-Scale Social Networks Using Snapshot Population Behaviour without Network Data

Authors: Antonia Godoy-Lorite, Nick S. Jones

Abstract: Population behaviours, such as voting and vaccination, depend on social networks. Social networks can differ depending on behaviour type and are typically hidden. However, we do often have large-scale behavioural data, albeit only snapshots taken at one timepoint. We present a method that jointly infers large-scale network structure and a networked model of human behaviour using only snapshot popu… ▽ More Population behaviours, such as voting and vaccination, depend on social networks. Social networks can differ depending on behaviour type and are typically hidden. However, we do often have large-scale behavioural data, albeit only snapshots taken at one timepoint. We present a method that jointly infers large-scale network structure and a networked model of human behaviour using only snapshot population behavioural data. This exploits the simplicity of a few parameter, geometric socio-demographic network model and a spin based model of behaviour. We illustrate, for the EU Referendum and two London Mayoral elections, how the model offers both prediction and the interpretation of our homophilic inclinations. Beyond offering the extraction of behaviour specific network structure from large-scale behavioural datasets, our approach yields a crude calculus linking inequalities and social preferences to behavioural outcomes. We give examples of potential network sensitive policies: how changes to income inequality, a social temperature and homophilic preferences might have reduced polarisation in a recent election. △ Less

Submitted 23 March, 2020; v1 submitted 16 March, 2020; originally announced March 2020.

Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog

Authors: Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, Rosalind Picard

Abstract: Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. These are critical shortcomings for applying RL to real-world problems where collecting data is expensive, and models must be tested offline before being deployed to interact with the environment -- e.g. systems that learn from human int… ▽ More Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. These are critical shortcomings for applying RL to real-world problems where collecting data is expensive, and models must be tested offline before being deployed to interact with the environment -- e.g. systems that learn from human interaction. Thus, we develop a novel class of off-policy batch RL algorithms, which are able to effectively learn offline, without exploring, from a fixed batch of human interaction data. We leverage models pre-trained on data as a strong prior, and use KL-control to penalize divergence from this prior during RL training. We also use dropout-based uncertainty estimates to lower bound the target Q-values as a more efficient alternative to Double Q-Learning. The algorithms are tested on the problem of open-domain dialog generation -- a challenging reinforcement learning problem with a 20,000-dimensional action space. Using our Way Off-Policy algorithm, we can extract multiple different reward functions post-hoc from collected human interaction data, and learn effectively from all of these. We test the real-world generalization of these systems by deploying them live to converse with humans in an open-domain setting, and demonstrate that our algorithm achieves significant improvements over prior methods in off-policy batch RL. △ Less

Submitted 8 July, 2019; v1 submitted 30 June, 2019; originally announced July 2019.

Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems

Authors: Asma Ghandeharioun, Judy Hanwen Shen, Natasha Jaques, Craig Ferguson, Noah Jones, Agata Lapedriza, Rosalind Picard

Abstract: Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; we then introduce a novel, model-agnostic, and dataset-agnostic method to approx… ▽ More Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; we then introduce a novel, model-agnostic, and dataset-agnostic method to approximate it. In particular, we propose a self-play scenario where the dialog system talks to itself and we calculate a combination of proxies such as sentiment and semantic coherence on the conversation trajectory. We show that this metric is capable of capturing the human-rated quality of a dialog model better than any automated metric known to-date, achieving a significant Pearson correlation (r>.7, p<.05). To investigate the strengths of this novel metric and interactive evaluation in comparison to state-of-the-art metrics and human evaluation of static conversations, we perform extended experiments with a set of models, including several that make novel improvements to recent hierarchical dialog generation architectures through sentiment and semantic knowledge distillation on the utterance level. Finally, we open-source the interactive evaluation platform we built and the dataset we collected to allow researchers to efficiently deploy and evaluate dialog models. △ Less

Submitted 3 November, 2019; v1 submitted 21 June, 2019; originally announced June 2019.

Comments: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

Automating chaos experiments in production

Authors: Ali Basiri, Lorin Hochstein, Nora Jones, Haley Tucker

Abstract: Distributed systems often face transient errors and localized component degradation and failure. Verifying that the overall system remains healthy in the face of such failures is challenging. At Netflix, we have built a platform for automatically generating and executing chaos experiments, which check how well the production system can handle component failures and slowdowns. This paper describes… ▽ More Distributed systems often face transient errors and localized component degradation and failure. Verifying that the overall system remains healthy in the face of such failures is challenging. At Netflix, we have built a platform for automatically generating and executing chaos experiments, which check how well the production system can handle component failures and slowdowns. This paper describes the platform and our experiences operating it. △ Less

Submitted 12 May, 2019; originally announced May 2019.

Comments: International Conference on Software Engineering (ICSE), 2019

CompEngine: a self-organizing, living library of time-series data

Authors: Ben D. Fulcher, Carl H. Lubba, Sarab S. Sethi, Nick S. Jones

Abstract: Modern biomedical applications often involve time-series data, from high-throughput phenotyping of model organisms, through to individual disease diagnosis and treatment using biomedical data streams. Data and tools for time-series analysis are developed and applied across the sciences and in industry, but meaningful cross-disciplinary interactions are limited by the challenge of identifying fruit… ▽ More Modern biomedical applications often involve time-series data, from high-throughput phenotyping of model organisms, through to individual disease diagnosis and treatment using biomedical data streams. Data and tools for time-series analysis are developed and applied across the sciences and in industry, but meaningful cross-disciplinary interactions are limited by the challenge of identifying fruitful connections. Here we introduce the web platform, CompEngine, a self-organizing, living library of time-series data that lowers the barrier to forming meaningful interdisciplinary connections between time series. Using a canonical feature-based representation, CompEngine places all time series in a common space, regardless of their origin, allowing users to upload their data and immediately explore interdisciplinary connections to other data with similar properties, and be alerted when similar data is uploaded in the future. In contrast to conventional databases, which are organized by assigned metadata, CompEngine incentivizes data sharing by automatically connecting experimental and theoretical scientists across disciplines based on the empirical structure of their data. CompEngine's growing library of interdisciplinary time-series data also facilitates comprehensively characterization of algorithm performance across diverse types of data, and can be used to empirically motivate the development of new time-series analysis algorithms. △ Less

Submitted 3 May, 2019; originally announced May 2019.

catch22: CAnonical Time-series CHaracteristics

Authors: Carl H Lubba, Sarab S Sethi, Philip Knaute, Simon R Schultz, Ben D Fulcher, Nick S Jones

Abstract: Capturing the dynamical properties of time series concisely as interpretable feature vectors can enable efficient clustering and classification for time-series applications across science and industry. Selecting an appropriate feature-based representation of time series for a given application can be achieved through systematic comparison across a comprehensive time-series feature library, such as… ▽ More Capturing the dynamical properties of time series concisely as interpretable feature vectors can enable efficient clustering and classification for time-series applications across science and industry. Selecting an appropriate feature-based representation of time series for a given application can be achieved through systematic comparison across a comprehensive time-series feature library, such as those in the hctsa toolbox. However, this approach is computationally expensive and involves evaluating many similar features, limiting the widespread adoption of feature-based representations of time series for real-world applications. In this work, we introduce a method to infer small sets of time-series features that (i) exhibit strong classification performance across a given collection of time-series problems, and (ii) are minimally redundant. Applying our method to a set of 93 time-series classification datasets (containing over 147000 time series) and using a filtered version of the hctsa feature library (4791 features), we introduce a generically useful set of 22 CAnonical Time-series CHaracteristics, catch22. This dimensionality reduction, from 4791 to 22, is associated with an approximately 1000-fold reduction in computation time and near linear scaling with time-series length, despite an average reduction in classification accuracy of just 7%. catch22 captures a diverse and interpretable signature of time series in terms of their properties, including linear and non-linear autocorrelation, successive differences, value distributions and outliers, and fluctuation scaling properties. We provide an efficient implementation of catch22, accessible from many programming environments, that facilitates feature-based time-series analysis for scientific, industrial, financial and medical applications using a common language of interpretable time-series properties. △ Less

Submitted 30 January, 2019; v1 submitted 29 January, 2019; originally announced January 2019.

Community detection in networks without observing edges

Authors: Till Hoffmann, Leto Peel, Renaud Lambiotte, Nick S. Jones

Abstract: We develop a Bayesian hierarchical model to identify communities in networks for which we do not observe the edges directly, but instead observe a series of interdependent signals for each of the nodes. Fitting the model provides an end-to-end community detection algorithm that does not extract information as a sequence of point estimates but propagates uncertainties from the raw data to the commu… ▽ More We develop a Bayesian hierarchical model to identify communities in networks for which we do not observe the edges directly, but instead observe a series of interdependent signals for each of the nodes. Fitting the model provides an end-to-end community detection algorithm that does not extract information as a sequence of point estimates but propagates uncertainties from the raw data to the community labels. Our approach naturally supports multiscale community detection as well as the selection of an optimal scale using model comparison. We study the properties of the algorithm using synthetic data and apply it to daily returns of constituents of the S&P100 index as well as climate data from US cities. △ Less

Submitted 11 February, 2020; v1 submitted 18 August, 2018; originally announced August 2018.

Comments: 16 pages, 7 figures

Journal ref: Science Advances 6(4) eaav1478, 2020

Co-occurrence simplicial complexes in mathematics: identifying the holes of knowledge

Authors: Vsevolod Salnikov, Daniele Cassese, Renaud Lambiotte, Nick S. Jones

Abstract: In the last years complex networks tools contributed to provide insights on the structure of research, through the study of collaboration, citation and co-occurrence networks. The network approach focuses on pairwise relationships, often compressing multidimensional data structures and inevitably losing information. In this paper we propose for the first time a simplicial complex approach to word… ▽ More In the last years complex networks tools contributed to provide insights on the structure of research, through the study of collaboration, citation and co-occurrence networks. The network approach focuses on pairwise relationships, often compressing multidimensional data structures and inevitably losing information. In this paper we propose for the first time a simplicial complex approach to word co-occurrences, providing a natural framework for the study of higher-order relations in the space of scientific knowledge. Using topological methods we explore the conceptual landscape of mathematical research, focusing on homological holes, regions with low connectivity in the simplicial structure. We find that homological holes are ubiquitous, which suggests that they capture some essential feature of research practice in mathematics. Holes die when a subset of their concepts appear in the same article, hence their death may be a sign of the creation of new knowledge, as we show with some examples. We find a positive relation between the dimension of a hole and the time it takes to be closed: larger holes may represent potential for important advances in the field because they separate conceptually distant areas. We also show that authors' conceptual entropy is positively related with their contribution to homological holes, suggesting that polymaths tend to be on the frontier of research. △ Less

Submitted 11 March, 2018; originally announced March 2018.

Comments: 24 pages, 12 figures

Design of a nickel-base superalloy using a neural network

Authors: B. D. Conduit, N. G. Jones, H. J. Stone, G. J. Conduit

Abstract: A new computational tool has been developed to model, discover, and optimize new alloys that simultaneously satisfy up to eleven physical criteria. An artificial neural network is trained from pre-existing materials data that enables the prediction of individual material properties both as a function of composition and heat treatment routine, which allows it to optimize the material properties to… ▽ More A new computational tool has been developed to model, discover, and optimize new alloys that simultaneously satisfy up to eleven physical criteria. An artificial neural network is trained from pre-existing materials data that enables the prediction of individual material properties both as a function of composition and heat treatment routine, which allows it to optimize the material properties to search for the material with properties most likely to exceed a target criteria. We design a new polycrystalline nickel-base superalloy with the optimal combination of cost, density, gamma' phase content and solvus, phase stability, fatigue life, yield stress, ultimate tensile strength, stress rupture, oxidation resistance, and tensile elongation. Experimental data demonstrates that the proposed alloy fulfills the computational predictions, possessing multiple physical properties, particularly oxidation resistance and yield stress, that exceed existing commercially available alloys. △ Less

Submitted 8 March, 2018; originally announced March 2018.

Journal ref: Materials & Design 131, 358 (2017)

Probabilistic design of a molybdenum-base alloy using a neural network

Authors: B. D. Conduit, N. G. Jones, H. J. Stone, G. J. Conduit

Abstract: An artificial intelligence tool is exploited to discover and characterize a new molybdenum-base alloy that is the most likely to simultaneously satisfy targets of cost, phase stability, precipitate content, yield stress, and hardness. Experimental testing demonstrates that the proposed alloy fulfils the computational predictions, and furthermore the physical properties exceed those of other commer… ▽ More An artificial intelligence tool is exploited to discover and characterize a new molybdenum-base alloy that is the most likely to simultaneously satisfy targets of cost, phase stability, precipitate content, yield stress, and hardness. Experimental testing demonstrates that the proposed alloy fulfils the computational predictions, and furthermore the physical properties exceed those of other commercially available Mo-base alloys for forging-die applications. △ Less

Submitted 2 March, 2018; originally announced March 2018.

Journal ref: Scripta Materialia 146, 82 (2018)

Automatic time-series phenotyping using massive feature extraction

Authors: Ben D Fulcher, Nick S Jones

Abstract: Across a far-reaching diversity of scientific and industrial applications, a general key problem involves relating the structure of time-series data to a meaningful outcome, such as detecting anomalous events from sensor recordings, or diagnosing patients from physiological time-series measurements like heart rate or brain activity. Currently, researchers must devote considerable effort manually d… ▽ More Across a far-reaching diversity of scientific and industrial applications, a general key problem involves relating the structure of time-series data to a meaningful outcome, such as detecting anomalous events from sensor recordings, or diagnosing patients from physiological time-series measurements like heart rate or brain activity. Currently, researchers must devote considerable effort manually devising, or searching for, properties of their time series that are suitable for the particular analysis problem at hand. Addressing this non-systematic and time-consuming procedure, here we introduce a new tool, hctsa, that selects interpretable and useful properties of time series automatically, by comparing implementations over 7700 time-series features drawn from diverse scientific literatures. Using two exemplar biological applications, we show how hctsa allows researchers to leverage decades of time-series research to quantify and understand informative structure in their time-series data. △ Less

Submitted 15 December, 2016; originally announced December 2016.

Journal ref: Cell Systems 5 (2017) 527

Report on the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3)

Authors: Daniel S. Katz, Sou-Cheng T. Choi, Kyle E. Niemeyer, James Hetherington, Frank Löffler, Dan Gunter, Ray Idaszak, Steven R. Brandt, Mark A. Miller, Sandra Gesing, Nick D. Jones, Nic Weber, Suresh Marru, Gabrielle Allen, Birgit Penzenstadler, Colin C. Venters, Ethan Davis, Lorraine Hwang, Ilian Todorov, Abani Patra, Miguel de Val-Borro

Abstract: This report records and discusses the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3). The report includes a description of the keynote presentation of the workshop, which served as an overview of sustainable scientific software. It also summarizes a set of lightning talks in which speakers highlighted to-the-point lessons and challenges pertaining to sustain… ▽ More This report records and discusses the Third Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE3). The report includes a description of the keynote presentation of the workshop, which served as an overview of sustainable scientific software. It also summarizes a set of lightning talks in which speakers highlighted to-the-point lessons and challenges pertaining to sustaining scientific software. The final and main contribution of the report is a summary of the discussions, future steps, and future organization for a set of self-organized working groups on topics including developing pathways to funding scientific software; constructing useful common metrics for crediting software stakeholders; identifying principles for sustainable software engineering design; reaching out to research software organizations around the world; and building communities for software sustainability. For each group, we include a point of contact and a landing page that can be used by those who want to join that group's future activities. The main challenge left by the workshop is to see if the groups will execute these activities that they have scheduled, and how the WSSSPE community can encourage this to happen. △ Less

Submitted 6 February, 2016; originally announced February 2016.