Showing 1–7 of 7 results for author: Barrett, L

Operationalizing a Threat Model for Red-Teaming Large Language Models (LLMs)

Authors: Apurv Verma, Satyapriya Krishna, Sebastian Gehrmann, Madhavan Seshadri, Anu Pradhan, Tom Ault, Leslie Barrett, David Rabinowitz, John Doucette, NhatHai Phan

Abstract: Creating secure and resilient applications with large language models (LLM) requires anticipating, adjusting to, and countering unforeseen threats. Red-teaming has emerged as a critical technique for identifying vulnerabilities in real-world LLM implementations. This paper presents a detailed threat model and provides a systematization of knowledge (SoK) of red-teaming attacks on LLMs. We develop… ▽ More Creating secure and resilient applications with large language models (LLM) requires anticipating, adjusting to, and countering unforeseen threats. Red-teaming has emerged as a critical technique for identifying vulnerabilities in real-world LLM implementations. This paper presents a detailed threat model and provides a systematization of knowledge (SoK) of red-teaming attacks on LLMs. We develop a taxonomy of attacks based on the stages of the LLM development and deployment process and extract various insights from previous research. In addition, we compile methods for defense and practical red-teaming strategies for practitioners. By delineating prominent attack motifs and shedding light on various entry points, this paper provides a framework for improving the security and robustness of LLM-based systems. △ Less

Submitted 20 July, 2024; originally announced July 2024.

Comments: Preprint. Under review

Variation is the Norm: Brain State Dynamics Evoked By Emotional Video Clips

Authors: Ashutosh Singh, Christiana Westlin, Hedwig Eisenbarth, Elizabeth A. Reynolds Losin, Jessica R. Andrews-Hanna, Tor D. Wager, Ajay B. Satpute, Lisa Feldman Barrett, Dana H. Brooks, Deniz Erdogmus

Abstract: For the last several decades, emotion research has attempted to identify a "biomarker" or consistent pattern of brain activity to characterize a single category of emotion (e.g., fear) that will remain consistent across all instances of that category, regardless of individual and context. In this study, we investigated variation rather than consistency during emotional experiences while people wat… ▽ More For the last several decades, emotion research has attempted to identify a "biomarker" or consistent pattern of brain activity to characterize a single category of emotion (e.g., fear) that will remain consistent across all instances of that category, regardless of individual and context. In this study, we investigated variation rather than consistency during emotional experiences while people watched video clips chosen to evoke instances of specific emotion categories. Specifically, we developed a sequential probabilistic approach to model the temporal dynamics in a participant's brain activity during video viewing. We characterized brain states during these clips as distinct state occupancy periods between state transitions in blood oxygen level dependent (BOLD) signal patterns. We found substantial variation in the state occupancy probability distributions across individuals watching the same video, supporting the hypothesis that when it comes to the brain correlates of emotional experience, variation may indeed be the norm. △ Less

Submitted 24 October, 2021; originally announced October 2021.

Enhancing Prediction Models for One-Year Mortality in Patients with Acute Myocardial Infarction and Post Myocardial Infarction Syndrome

Authors: Seyedeh Neelufar Payrovnaziri, Laura A. Barrett, Daniel Bis, Jiang Bian, Zhe He

Abstract: Predicting the risk of mortality for patients with acute myocardial infarction (AMI) using electronic health records (EHRs) data can help identify risky patients who might need more tailored care. In our previous work, we built computational models to predict one-year mortality of patients admitted to an intensive care unit (ICU) with AMI or post myocardial infarction syndrome. Our prior work only… ▽ More Predicting the risk of mortality for patients with acute myocardial infarction (AMI) using electronic health records (EHRs) data can help identify risky patients who might need more tailored care. In our previous work, we built computational models to predict one-year mortality of patients admitted to an intensive care unit (ICU) with AMI or post myocardial infarction syndrome. Our prior work only used the structured clinical data from MIMIC-III, a publicly available ICU clinical database. In this study, we enhanced our work by adding the word embedding features from free-text discharge summaries. Using a richer set of features resulted in significant improvement in the performance of our deep learning models. The average accuracy of our deep learning models was 92.89% and the average F-measure was 0.928. We further reported the impact of different combinations of features extracted from structured and/or unstructured data on the performance of the deep learning models. △ Less

Submitted 28 April, 2019; originally announced April 2019.

Building Computational Models to Predict One-Year Mortality in ICU Patients with Acute Myocardial Infarction and Post Myocardial Infarction Syndrome

Authors: Laura A. Barrett, Seyedeh Neelufar Payrovnaziri, Jiang Bian, Zhe He

Abstract: Heart disease remains the leading cause of death in the United States. Compared with risk assessment guidelines that require manual calculation of scores, machine learning-based prediction for disease outcomes such as mortality can be utilized to save time and improve prediction accuracy. This study built and evaluated various machine learning models to predict one-year mortality in patients diagn… ▽ More Heart disease remains the leading cause of death in the United States. Compared with risk assessment guidelines that require manual calculation of scores, machine learning-based prediction for disease outcomes such as mortality can be utilized to save time and improve prediction accuracy. This study built and evaluated various machine learning models to predict one-year mortality in patients diagnosed with acute myocardial infarction or post myocardial infarction syndrome in the MIMIC-III database. The results of the best performing shallow prediction models were compared to a deep feedforward neural network (Deep FNN) with back propagation. We included a cohort of 5436 admissions. Six datasets were developed and compared. The models applying Logistic Model Trees (LMT) and Simple Logistic algorithms to the combined dataset resulted in the highest prediction accuracy at 85.12% and the highest AUC at .901. In addition, other factors were observed to have an impact on outcomes as well. △ Less

Submitted 12 December, 2018; originally announced December 2018.

Civil Asset Forfeiture: A Judicial Perspective

Authors: Leslie Barrett, Wayne Krug, Zefu Lu, Karin D. Martin, Roberto Martin, Alexandra Ortan, Anu Pradhan, Alexander Sherman, Michael W. Sherman, Ryon Smey, Trent Wenzel

Abstract: Civil Asset Forfeiture (CAF) is a longstanding and controversial legal process viewed on the one hand as a powerful tool for combating drug crimes and on the other hand as a violation of the rights of US citizens. Data used to support both sides of the controversy to date has come from government sources representing records of the events at the time of occurrence. Court dockets represent litigati… ▽ More Civil Asset Forfeiture (CAF) is a longstanding and controversial legal process viewed on the one hand as a powerful tool for combating drug crimes and on the other hand as a violation of the rights of US citizens. Data used to support both sides of the controversy to date has come from government sources representing records of the events at the time of occurrence. Court dockets represent litigation events initiated following the forfeiture, however, and can thus provide a new perspective on the CAF legal process. This paper will show new evidence supporting existing claims about the growth of the practice and bias in its application based on the quantitative analysis of data derived from these court cases. △ Less

Submitted 5 October, 2017; originally announced October 2017.

Comments: Presented at the Data For Good Exchange 2017

Way to Go! Automatic Optimization of Wayfinding Design

Authors: Haikun Huang, Ni-Ching Lin, Lorenzo Barrett, Darian Springer, Hsueh-Cheng Wang, Marc Pomplun, Lap-Fai Yu

Abstract: Wayfinding signs play an important role in guiding users to navigate in a virtual environment and in helping pedestrians to find their ways in a real-world architectural site. Conventionally, the wayfinding design of a virtual environment is created manually, so as the wayfinding design of a real-world architectural site. The many possible navigation scenarios, as well as the interplay between sig… ▽ More Wayfinding signs play an important role in guiding users to navigate in a virtual environment and in helping pedestrians to find their ways in a real-world architectural site. Conventionally, the wayfinding design of a virtual environment is created manually, so as the wayfinding design of a real-world architectural site. The many possible navigation scenarios, as well as the interplay between signs and human navigation, can make the manual design process overwhelming and non-trivial. As a result, creating a wayfinding design for a typical layout can take months to several years. In this paper, we introduce the Way to Go! approach for automatically generating a wayfinding design for a given layout. The designer simply has to specify some navigation scenarios; our approach will automatically generate an optimized wayfinding design with signs properly placed considering human agents' visibility and possibility of making mistakes during a navigation. We demonstrate the effectiveness of our approach in generating wayfinding designs for different layouts such as a train station, a downtown and a canyon. We evaluate our results by comparing different wayfinding designs and show that our optimized wayfinding design can guide pedestrians to their destinations effectively and efficiently. Our approach can also help the designer visualize the accessibility of a destination from different locations, and correct any "blind zone" with additional signs. △ Less

Submitted 27 June, 2017; originally announced June 2017.

Formalising the multidimensional nature of social networks

Authors: David Lusseau, Louise Barrett, S. Peter Henzi

Abstract: Individuals interact with conspecifics in a number of behavioural contexts or dimensions. Here, we formalise this by considering a social network between n individuals interacting in b behavioural dimensions as a nxnxb multidimensional object. In addition, we propose that the topology of this object is driven by individual needs to reduce uncertainty about the outcomes of interactions in one or mo… ▽ More Individuals interact with conspecifics in a number of behavioural contexts or dimensions. Here, we formalise this by considering a social network between n individuals interacting in b behavioural dimensions as a nxnxb multidimensional object. In addition, we propose that the topology of this object is driven by individual needs to reduce uncertainty about the outcomes of interactions in one or more dimension. The proposal grounds social network dynamics and evolution in individual selection processes and allows us to define the uncertainty of the social network as the joint entropy of its constituent interaction networks. In support of these propositions we use simulations and natural 'knock-outs' in a free-ranging baboon troop to show (i) that such an object can display a small-world state and (ii) that, as predicted, changes in interactions after social perturbations lead to a more certain social network, in which the outcomes of interactions are easier for members to predict. This new formalisation of social networks provides a framework within which to predict network dynamics and evolution under the assumption that it is driven by individuals seeking to reduce the uncertainty of their social environment. △ Less

Submitted 19 January, 2011; originally announced January 2011.

Comments: 16 pages, 4 figures

Journal ref: 2012: Philosophical Transactions of the Royal Society of London B 367(1599):2108-2118