-
On the Connection Between Non-negative Matrix Factorization and Latent Dirichlet Allocation
Authors:
Benedikt Geiger,
Peter J. Park
Abstract:
Non-negative matrix factorization with the generalized Kullback-Leibler divergence (NMF) and latent Dirichlet allocation (LDA) are two popular approaches for dimensionality reduction of non-negative data. Here, we show that NMF with $\ell_1$ normalization constraints on the columns of both matrices of the decomposition and a Dirichlet prior on the columns of one matrix is equivalent to LDA. To sho…
▽ More
Non-negative matrix factorization with the generalized Kullback-Leibler divergence (NMF) and latent Dirichlet allocation (LDA) are two popular approaches for dimensionality reduction of non-negative data. Here, we show that NMF with $\ell_1$ normalization constraints on the columns of both matrices of the decomposition and a Dirichlet prior on the columns of one matrix is equivalent to LDA. To show this, we demonstrate that explicitly accounting for the scaling ambiguity of NMF by adding $\ell_1$ normalization constraints to the optimization problem allows a joint update of both matrices in the widely used multiplicative updates (MU) algorithm. When both of the matrices are normalized, the joint MU algorithm leads to probabilistic latent semantic analysis (PLSA), which is LDA without a Dirichlet prior. Our approach of deriving joint updates for NMF also reveals that a Lasso penalty on one matrix together with an $\ell_1$ normalization constraint on the other matrix is insufficient to induce any sparsity.
△ Less
Submitted 30 May, 2024;
originally announced May 2024.
-
Heterogeneous Image-based Classification Using Distributional Data Analysis
Authors:
Alec Reinhardt,
Newsha Nikzad,
Raven J. Hollis,
Galia Jacobson,
Millicent A. Roach,
Mohamed Badawy,
Peter Chul Park,
Laura Beretta,
Prasun K Jalal,
David T. Fuentes,
Eugene J. Koay,
Suprateek Kundu
Abstract:
Diagnostic imaging has gained prominence as potential biomarkers for early detection and diagnosis in a diverse array of disorders including cancer. However, existing methods routinely face challenges arising from various factors such as image heterogeneity. We develop a novel imaging-based distributional data analysis (DDA) approach that incorporates the probability (quantile) distribution of the…
▽ More
Diagnostic imaging has gained prominence as potential biomarkers for early detection and diagnosis in a diverse array of disorders including cancer. However, existing methods routinely face challenges arising from various factors such as image heterogeneity. We develop a novel imaging-based distributional data analysis (DDA) approach that incorporates the probability (quantile) distribution of the pixel-level features as covariates. The proposed approach uses a smoothed quantile distribution (via a suitable basis representation) as functional predictors in a scalar-on-functional quantile regression model. Some distinctive features of the proposed approach include the ability to: (i) account for heterogeneity within the image; (ii) incorporate granular information spanning the entire distribution; and (iii) tackle variability in image sizes for unregistered images in cancer applications. Our primary goal is risk prediction in Hepatocellular carcinoma that is achieved via predicting the change in tumor grades at post-diagnostic visits using pre-diagnostic enhancement pattern mapping (EPM) images of the liver. Along the way, the proposed DDA approach is also used for case versus control diagnosis and risk stratification objectives. Our analysis reveals that when coupled with global structural radiomics features derived from the corresponding T1-MRI scans, the proposed smoothed quantile distributions derived from EPM images showed considerable improvements in sensitivity and comparable specificity in contrast to classification based on routinely used summary measures that do not account for image heterogeneity. Given that there are limited predictive modeling approaches based on heterogeneous images in cancer, the proposed method is expected to provide considerable advantages in image-based early detection and risk prediction.
△ Less
Submitted 11 March, 2024;
originally announced March 2024.
-
Wisdom of the Silicon Crowd: LLM Ensemble Prediction Capabilities Rival Human Crowd Accuracy
Authors:
Philipp Schoenegger,
Indre Tuminauskaite,
Peter S. Park,
Philip E. Tetlock
Abstract:
Human forecasting accuracy in practice relies on the 'wisdom of the crowd' effect, in which predictions about future events are significantly improved by aggregating across a crowd of individual forecasters. Past work on the forecasting ability of large language models (LLMs) suggests that frontier LLMs, as individual forecasters, underperform compared to the gold standard of a human crowd forecas…
▽ More
Human forecasting accuracy in practice relies on the 'wisdom of the crowd' effect, in which predictions about future events are significantly improved by aggregating across a crowd of individual forecasters. Past work on the forecasting ability of large language models (LLMs) suggests that frontier LLMs, as individual forecasters, underperform compared to the gold standard of a human crowd forecasting tournament aggregate. In Study 1, we expand this research by using an LLM ensemble approach consisting of a crowd of twelve LLMs. We compare the aggregated LLM predictions on 31 binary questions to that of a crowd of 925 human forecasters from a three-month forecasting tournament. Our preregistered main analysis shows that the LLM crowd outperforms a simple no-information benchmark and is not statistically different from the human crowd. In exploratory analyses, we find that these two approaches are equivalent with respect to medium-effect-size equivalence bounds. We also observe an acquiescence effect, with mean model predictions being significantly above 50%, despite an almost even split of positive and negative resolutions. Moreover, in Study 2, we test whether LLM predictions (of GPT-4 and Claude 2) can be improved by drawing on human cognitive output. We find that both models' forecasting accuracy benefits from exposure to the median human prediction as information, improving accuracy by between 17% and 28%: though this leads to less accurate predictions than simply averaging human and machine forecasts. Our results suggest that LLMs can achieve forecasting accuracy rivaling that of human crowd forecasting tournaments: via the simple, practically applicable method of forecast aggregation. This replicates the 'wisdom of the crowd' effect for LLMs, and opens up their use for a variety of applications throughout society.
△ Less
Submitted 22 July, 2024; v1 submitted 29 February, 2024;
originally announced February 2024.
-
AI-Augmented Predictions: LLM Assistants Improve Human Forecasting Accuracy
Authors:
Philipp Schoenegger,
Peter S. Park,
Ezra Karger,
Sean Trott,
Philip E. Tetlock
Abstract:
Large language models (LLMs) match and sometimes exceeding human performance in many domains. This study explores the potential of LLMs to augment human judgement in a forecasting task. We evaluate the effect on human forecasters of two LLM assistants: one designed to provide high-quality ("superforecasting") advice, and the other designed to be overconfident and base-rate neglecting, thus providi…
▽ More
Large language models (LLMs) match and sometimes exceeding human performance in many domains. This study explores the potential of LLMs to augment human judgement in a forecasting task. We evaluate the effect on human forecasters of two LLM assistants: one designed to provide high-quality ("superforecasting") advice, and the other designed to be overconfident and base-rate neglecting, thus providing noisy forecasting advice. We compare participants using these assistants to a control group that received a less advanced model that did not provide numerical predictions or engaged in explicit discussion of predictions. Participants (N = 991) answered a set of six forecasting questions and had the option to consult their assigned LLM assistant throughout. Our preregistered analyses show that interacting with each of our frontier LLM assistants significantly enhances prediction accuracy by between 24 percent and 28 percent compared to the control group. Exploratory analyses showed a pronounced outlier effect in one forecasting item, without which we find that the superforecasting assistant increased accuracy by 41 percent, compared with 29 percent for the noisy assistant. We further examine whether LLM forecasting augmentation disproportionately benefits less skilled forecasters, degrades the wisdom-of-the-crowd by reducing prediction diversity, or varies in effectiveness with question difficulty. Our data do not consistently support these hypotheses. Our results suggest that access to a frontier LLM assistant, even a noisy one, can be a helpful decision aid in cognitively demanding tasks compared to a less powerful model that does not provide specific forecasting advice. However, the effects of outliers suggest that further research into the robustness of this pattern is needed.
△ Less
Submitted 22 August, 2024; v1 submitted 12 February, 2024;
originally announced February 2024.
-
Development and validation of an artificial intelligence model to accurately predict spinopelvic parameters
Authors:
Edward S. Harake,
Joseph R. Linzey,
Cheng Jiang,
Rushikesh S. Joshi,
Mark M. Zaki,
Jaes C. Jones,
Siri S. Khalsa,
John H. Lee,
Zachary Wilseck,
Jacob R. Joseph,
Todd C. Hollon,
Paul Park
Abstract:
Objective. Achieving appropriate spinopelvic alignment has been shown to be associated with improved clinical symptoms. However, measurement of spinopelvic radiographic parameters is time-intensive and interobserver reliability is a concern. Automated measurement tools have the promise of rapid and consistent measurements, but existing tools are still limited by some degree of manual user-entry re…
▽ More
Objective. Achieving appropriate spinopelvic alignment has been shown to be associated with improved clinical symptoms. However, measurement of spinopelvic radiographic parameters is time-intensive and interobserver reliability is a concern. Automated measurement tools have the promise of rapid and consistent measurements, but existing tools are still limited by some degree of manual user-entry requirements. This study presents a novel artificial intelligence (AI) tool called SpinePose that automatically predicts spinopelvic parameters with high accuracy without the need for manual entry.
Methods. SpinePose was trained and validated on 761 sagittal whole-spine X-rays to predict sagittal vertical axis (SVA), pelvic tilt (PT), pelvic incidence (PI), sacral slope (SS), lumbar lordosis (LL), T1-pelvic angle (T1PA), and L1-pelvic angle (L1PA). A separate test set of 40 X-rays was labeled by 4 reviewers, including fellowship-trained spine surgeons and a fellowship-trained radiologist with neuroradiology subspecialty certification. Median errors relative to the most senior reviewer were calculated to determine model accuracy on test images. Intraclass correlation coefficients (ICC) were used to assess inter-rater reliability.
Results. SpinePose exhibited the following median (interquartile range) parameter errors: SVA: 2.2(2.3)mm, p=0.93; PT: 1.3(1.2)°, p=0.48; SS: 1.7(2.2)°, p=0.64; PI: 2.2(2.1)°, p=0.24; LL: 2.6(4.0)°, p=0.89; T1PA: 1.1(0.9)°, p=0.42; and L1PA: 1.4(1.6)°, p=0.49. Model predictions also exhibited excellent reliability at all parameters (ICC: 0.91-1.0).
Conclusions. SpinePose accurately predicted spinopelvic parameters with excellent reliability comparable to fellowship-trained spine surgeons and neuroradiologists. Utilization of predictive AI tools in spinal imaging can substantially aid in patient selection and surgical planning.
△ Less
Submitted 8 February, 2024;
originally announced February 2024.
-
Large Language Model Prediction Capabilities: Evidence from a Real-World Forecasting Tournament
Authors:
Philipp Schoenegger,
Peter S. Park
Abstract:
Accurately predicting the future would be an important milestone in the capabilities of artificial intelligence. However, research on the ability of large language models to provide probabilistic predictions about future events remains nascent. To empirically test this ability, we enrolled OpenAI's state-of-the-art large language model, GPT-4, in a three-month forecasting tournament hosted on the…
▽ More
Accurately predicting the future would be an important milestone in the capabilities of artificial intelligence. However, research on the ability of large language models to provide probabilistic predictions about future events remains nascent. To empirically test this ability, we enrolled OpenAI's state-of-the-art large language model, GPT-4, in a three-month forecasting tournament hosted on the Metaculus platform. The tournament, running from July to October 2023, attracted 843 participants and covered diverse topics including Big Tech, U.S. politics, viral outbreaks, and the Ukraine conflict. Focusing on binary forecasts, we show that GPT-4's probabilistic forecasts are significantly less accurate than the median human-crowd forecasts. We find that GPT-4's forecasts did not significantly differ from the no-information forecasting strategy of assigning a 50% probability to every question. We explore a potential explanation, that GPT-4 might be predisposed to predict probabilities close to the midpoint of the scale, but our data do not support this hypothesis. Overall, we find that GPT-4 significantly underperforms in real-world predictive tasks compared to median human-crowd forecasts. A potential explanation for this underperformance is that in real-world forecasting tournaments, the true answers are genuinely unknown at the time of prediction; unlike in other benchmark tasks like professional exams or time series forecasting, where strong performance may at least partly be due to the answers being memorized from the training data. This makes real-world forecasting tournaments an ideal environment for testing the generalized reasoning and prediction capabilities of artificial intelligence going forward.
△ Less
Submitted 17 October, 2023;
originally announced October 2023.
-
Divide-and-Conquer Dynamics in AI-Driven Disempowerment
Authors:
Peter S. Park,
Max Tegmark
Abstract:
AI companies are attempting to create AI systems that outperform humans at most economically valuable work. Current AI models are already automating away the livelihoods of some artists, actors, and writers. But there is infighting between those who prioritize current harms and future harms. We construct a game-theoretic model of conflict to study the causes and consequences of this disunity. Our…
▽ More
AI companies are attempting to create AI systems that outperform humans at most economically valuable work. Current AI models are already automating away the livelihoods of some artists, actors, and writers. But there is infighting between those who prioritize current harms and future harms. We construct a game-theoretic model of conflict to study the causes and consequences of this disunity. Our model also helps explain why throughout history, stakeholders sharing a common threat have found it advantageous to unite against it, and why the common threat has in turn found it advantageous to divide and conquer.
Under realistic parameter assumptions, our model makes several predictions that find preliminary corroboration in the historical-empirical record. First, current victims of AI-driven disempowerment need the future victims to realize that their interests are also under serious and imminent threat, so that future victims are incentivized to support current victims in solidarity. Second, the movement against AI-driven disempowerment can become more united, and thereby more likely to prevail, if members believe that their efforts will be successful as opposed to futile. Finally, the movement can better unite and prevail if its members are less myopic. Myopic members prioritize their future well-being less than their present well-being, and are thus disinclined to solidarily support current victims today at personal cost, even if this is necessary to counter the shared threat of AI-driven disempowerment.
△ Less
Submitted 18 December, 2023; v1 submitted 9 October, 2023;
originally announced October 2023.
-
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Authors:
Peter S. Park,
Simon Goldstein,
Aidan O'Gara,
Michael Chen,
Dan Hendrycks
Abstract:
This paper argues that a range of current AI systems have learned how to deceive humans. We define deception as the systematic inducement of false beliefs in the pursuit of some outcome other than the truth. We first survey empirical examples of AI deception, discussing both special-use AI systems (including Meta's CICERO) built for specific competitive situations, and general-purpose AI systems (…
▽ More
This paper argues that a range of current AI systems have learned how to deceive humans. We define deception as the systematic inducement of false beliefs in the pursuit of some outcome other than the truth. We first survey empirical examples of AI deception, discussing both special-use AI systems (including Meta's CICERO) built for specific competitive situations, and general-purpose AI systems (such as large language models). Next, we detail several risks from AI deception, such as fraud, election tampering, and losing control of AI systems. Finally, we outline several potential solutions to the problems posed by AI deception: first, regulatory frameworks should subject AI systems that are capable of deception to robust risk-assessment requirements; second, policymakers should implement bot-or-not laws; and finally, policymakers should prioritize the funding of relevant research, including tools to detect AI deception and to make AI systems less deceptive. Policymakers, researchers, and the broader public should work proactively to prevent AI deception from destabilizing the shared foundations of our society.
△ Less
Submitted 28 August, 2023;
originally announced August 2023.
-
Devising and Detecting Phishing: Large Language Models vs. Smaller Human Models
Authors:
Fredrik Heiding,
Bruce Schneier,
Arun Vishwanath,
Jeremy Bernstein,
Peter S. Park
Abstract:
AI programs, built using large language models, make it possible to automatically create phishing emails based on a few data points about a user. They stand in contrast to traditional phishing emails that hackers manually design using general rules gleaned from experience. The V-Triad is an advanced set of rules for manually designing phishing emails to exploit our cognitive heuristics and biases.…
▽ More
AI programs, built using large language models, make it possible to automatically create phishing emails based on a few data points about a user. They stand in contrast to traditional phishing emails that hackers manually design using general rules gleaned from experience. The V-Triad is an advanced set of rules for manually designing phishing emails to exploit our cognitive heuristics and biases. In this study, we compare the performance of phishing emails created automatically by GPT-4 and manually using the V-Triad. We also combine GPT-4 with the V-Triad to assess their combined potential. A fourth group, exposed to generic phishing emails, was our control group. We utilized a factorial approach, sending emails to 112 randomly selected participants recruited for the study. The control group emails received a click-through rate between 19-28%, the GPT-generated emails 30-44%, emails generated by the V-Triad 69-79%, and emails generated by GPT and the V-Triad 43-81%. Each participant was asked to explain why they pressed or did not press a link in the email. These answers often contradict each other, highlighting the need for personalized content. The cues that make one person avoid phishing emails make another person fall for them. Next, we used four popular large language models (GPT, Claude, PaLM, and LLaMA) to detect the intention of phishing emails and compare the results to human detection. The language models demonstrated a strong ability to detect malicious intent, even in non-obvious phishing emails. They sometimes surpassed human detection, although often being slightly less accurate than humans. Finally, we make an analysis of the economic aspects of AI-enabled phishing attacks, showing how large language models can increase the incentives of phishing and spear phishing by reducing their costs.
△ Less
Submitted 30 November, 2023; v1 submitted 23 August, 2023;
originally announced August 2023.
-
Performance Evaluation of Transport Protocols and Roadmap to a High-Performance Transport Design for Immersive Applications
Authors:
Inayat Ali,
Seungwoo Hong,
Pyung-koo Park,
Tae Yeon Kim
Abstract:
Immersive technologies such as virtual reality (VR), augmented reality (AR), and holograms will change users' digital experience. These immersive technologies have a multitude of applications, including telesurgeries, teleconferencing, Internet shopping, computer games, etc. Holographic-type communication (HTC) is a type of augmented reality media that provides an immersive experience to Internet…
▽ More
Immersive technologies such as virtual reality (VR), augmented reality (AR), and holograms will change users' digital experience. These immersive technologies have a multitude of applications, including telesurgeries, teleconferencing, Internet shopping, computer games, etc. Holographic-type communication (HTC) is a type of augmented reality media that provides an immersive experience to Internet users. However, HTC has different characteristics and network requirements, and the existing network architecture and transport protocols may not be able to cope with the stringent network requirements of HTC. Therefore, in this paper, we provide an in-depth and critical study of the transport protocols for HTC. We also discuss the characteristics and the network requirements for HTC. Based on the performance evaluation of the existing transport protocols, we propose a roadmap to design new high-performance transport protocols for immersive applications.
△ Less
Submitted 6 August, 2024; v1 submitted 29 June, 2023;
originally announced June 2023.
-
Diminished Diversity-of-Thought in a Standard Large Language Model
Authors:
Peter S. Park,
Philipp Schoenegger,
Chongyang Zhu
Abstract:
We test whether Large Language Models (LLMs) can be used to simulate human participants in social-science studies. To do this, we run replications of 14 studies from the Many Labs 2 replication project with OpenAI's text-davinci-003 model, colloquially known as GPT3.5. Based on our pre-registered analyses, we find that among the eight studies we could analyse, our GPT sample replicated 37.5% of th…
▽ More
We test whether Large Language Models (LLMs) can be used to simulate human participants in social-science studies. To do this, we run replications of 14 studies from the Many Labs 2 replication project with OpenAI's text-davinci-003 model, colloquially known as GPT3.5. Based on our pre-registered analyses, we find that among the eight studies we could analyse, our GPT sample replicated 37.5% of the original results and 37.5% of the Many Labs 2 results. However, we were unable to analyse the remaining six studies due to an unexpected phenomenon we call the "correct answer" effect. Different runs of GPT3.5 answered nuanced questions probing political orientation, economic preference, judgement, and moral philosophy with zero or near-zero variation in responses: with the supposedly "correct answer." In one exploratory follow-up study, we found that a "correct answer" was robust to changing the demographic details that precede the prompt. In another, we found that most but not all "correct answers" were robust to changing the order of answer choices. One of our most striking findings occurred in our replication of the Moral Foundations Theory survey results, where we found GPT3.5 identifying as a political conservative in 99.6% of the cases, and as a liberal in 99.3% of the cases in the reverse-order condition. However, both self-reported 'GPT conservatives' and 'GPT liberals' showed right-leaning moral foundations. Our results cast doubts on the validity of using LLMs as a general replacement for human participants in the social sciences. Our results also raise concerns that a hypothetical AI-led future may be subject to a diminished diversity-of-thought.
△ Less
Submitted 13 September, 2023; v1 submitted 13 February, 2023;
originally announced February 2023.
-
Discovering Exfiltration Paths Using Reinforcement Learning with Attack Graphs
Authors:
Tyler Cody,
Abdul Rahman,
Christopher Redino,
Lanxiao Huang,
Ryan Clark,
Akshay Kakkar,
Deepak Kushwaha,
Paul Park,
Peter Beling,
Edward Bowen
Abstract:
Reinforcement learning (RL), in conjunction with attack graphs and cyber terrain, are used to develop reward and state associated with determination of optimal paths for exfiltration of data in enterprise networks. This work builds on previous crown jewels (CJ) identification that focused on the target goal of computing optimal paths that adversaries may traverse toward compromising CJs or hosts w…
▽ More
Reinforcement learning (RL), in conjunction with attack graphs and cyber terrain, are used to develop reward and state associated with determination of optimal paths for exfiltration of data in enterprise networks. This work builds on previous crown jewels (CJ) identification that focused on the target goal of computing optimal paths that adversaries may traverse toward compromising CJs or hosts within their proximity. This work inverts the previous CJ approach based on the assumption that data has been stolen and now must be quietly exfiltrated from the network. RL is utilized to support the development of a reward function based on the identification of those paths where adversaries desire reduced detection. Results demonstrate promising performance for a sizable network environment.
△ Less
Submitted 25 April, 2022; v1 submitted 28 January, 2022;
originally announced January 2022.
-
Enhanced Frame and Event-Based Simulator and Event-Based Video Interpolation Network
Authors:
Adam Radomski,
Andreas Georgiou,
Thomas Debrunner,
Chenghan Li,
Luca Longinotti,
Minwon Seo,
Moosung Kwak,
Chang-Woo Shin,
Paul K. J. Park,
Hyunsurk Eric Ryu,
Kynan Eng
Abstract:
Fast neuromorphic event-based vision sensors (Dynamic Vision Sensor, DVS) can be combined with slower conventional frame-based sensors to enable higher-quality inter-frame interpolation than traditional methods relying on fixed motion approximations using e.g. optical flow. In this work we present a new, advanced event simulator that can produce realistic scenes recorded by a camera rig with an ar…
▽ More
Fast neuromorphic event-based vision sensors (Dynamic Vision Sensor, DVS) can be combined with slower conventional frame-based sensors to enable higher-quality inter-frame interpolation than traditional methods relying on fixed motion approximations using e.g. optical flow. In this work we present a new, advanced event simulator that can produce realistic scenes recorded by a camera rig with an arbitrary number of sensors located at fixed offsets. It includes a new configurable frame-based image sensor model with realistic image quality reduction effects, and an extended DVS model with more accurate characteristics. We use our simulator to train a novel reconstruction model designed for end-to-end reconstruction of high-fps video. Unlike previously published methods, our method does not require the frame and DVS cameras to have the same optics, positions, or camera resolutions. It is also not limited to objects a fixed distance from the sensor. We show that data generated by our simulator can be used to train our new model, leading to reconstructed images on public datasets of equivalent or better quality than the state of the art. We also show our sensor generalizing to data recorded by real sensors.
△ Less
Submitted 17 December, 2021;
originally announced December 2021.
-
Crown Jewels Analysis using Reinforcement Learning with Attack Graphs
Authors:
Rohit Gangupantulu,
Tyler Cody,
Abdul Rahman,
Christopher Redino,
Ryan Clark,
Paul Park
Abstract:
Cyber attacks pose existential threats to nations and enterprises. Current practice favors piece-wise analysis using threat-models in the stead of rigorous cyber terrain analysis and intelligence preparation of the battlefield. Automated penetration testing using reinforcement learning offers a new and promising approach for developing methodologies that are driven by network structure and cyber t…
▽ More
Cyber attacks pose existential threats to nations and enterprises. Current practice favors piece-wise analysis using threat-models in the stead of rigorous cyber terrain analysis and intelligence preparation of the battlefield. Automated penetration testing using reinforcement learning offers a new and promising approach for developing methodologies that are driven by network structure and cyber terrain, that can be later interpreted in terms of threat-models, but that are principally network-driven analyses. This paper presents a novel method for crown jewel analysis termed CJA-RL that uses reinforcement learning to identify key terrain and avenues of approach for exploiting crown jewels. In our experiment, CJA-RL identified ideal entry points, choke points, and pivots for exploiting a network with multiple crown jewels, exemplifying how CJA-RL and reinforcement learning for penetration testing generally can benefit computer network operations workflows.
△ Less
Submitted 20 August, 2021;
originally announced August 2021.
-
Using Cyber Terrain in Reinforcement Learning for Penetration Testing
Authors:
Rohit Gangupantulu,
Tyler Cody,
Paul Park,
Abdul Rahman,
Logan Eisenbeiser,
Dan Radke,
Ryan Clark
Abstract:
Reinforcement learning (RL) has been applied to attack graphs for penetration testing, however, trained agents do not reflect reality because the attack graphs lack operational nuances typically captured within the intelligence preparation of the battlefield (IPB) that include notions of (cyber) terrain. In particular, current practice constructs attack graphs exclusively using the Common Vulnerab…
▽ More
Reinforcement learning (RL) has been applied to attack graphs for penetration testing, however, trained agents do not reflect reality because the attack graphs lack operational nuances typically captured within the intelligence preparation of the battlefield (IPB) that include notions of (cyber) terrain. In particular, current practice constructs attack graphs exclusively using the Common Vulnerability Scoring System (CVSS) and its components. We present methods for constructing attack graphs using notions from IPB on cyber terrain analysis of obstacles, avenues of approach, key terrain, observation and fields of fire, and cover and concealment. We demonstrate our methods on an example where firewalls are treated as obstacles and represented in (1) the reward space and (2) the state dynamics. We show that terrain analysis can be used to bring realism to attack graphs for RL.
△ Less
Submitted 4 August, 2022; v1 submitted 16 August, 2021;
originally announced August 2021.
-
Wireless for Control: Over-the-Air Controller
Authors:
Pangun Park,
Piergiuseppe Di Marco,
Carlo Fischione
Abstract:
In closed-loop wireless control systems, the state-of-the-art approach prescribes that a controller receives by wireless communications the individual sensor measurements, and then sends the computed control signal to the actuators. We propose an over-the-air controller scheme where all sensors attached to the plant simultaneously transmit scaled sensing signals directly to the actuator; then the…
▽ More
In closed-loop wireless control systems, the state-of-the-art approach prescribes that a controller receives by wireless communications the individual sensor measurements, and then sends the computed control signal to the actuators. We propose an over-the-air controller scheme where all sensors attached to the plant simultaneously transmit scaled sensing signals directly to the actuator; then the feedback control signal is computed partially over the air and partially by a scaling operation at the actuator. Such over-the-air controller essentially adopts the over-the-air computation concept to compute the control signal for closed-loop wireless control systems. In contrast to the state-of-the-art sensor-to-controller and controller-to-actuator communication approach, the over-the-air controller exploits the superposition properties of multiple-access wireless channels to complete the communication and computation of a large number of sensing signals in a single communication resource unit. Therefore, the proposed scheme can obtain significant benefits in terms of low actuation delay and low wireless resource utilization by a simple network architecture that does not require a dedicated controller. Numerical results show that our proposed over-the-air controller achieves a huge widening of the stability region in terms of sampling time and delay, and a significant reduction of the computation error of the control signal.
△ Less
Submitted 27 May, 2021;
originally announced May 2021.
-
Wireless Avionics Intra-Communications: A Survey of Benefits, Challenges, and Solutions
Authors:
Pangun Park,
Piergiuseppe Di Marco,
Junghyo Nah,
Carlo Fischione
Abstract:
In the aeronautics industry, wireless avionics intra-communications have a tremendous potential to improve efficiency and flexibility while reducing the weight, fuel consumption, and maintenance costs over traditional wired avionics systems. This survey starts with an overview of the major benefits and opportunities in the deployment of wireless technologies for critical applications of an aircraf…
▽ More
In the aeronautics industry, wireless avionics intra-communications have a tremendous potential to improve efficiency and flexibility while reducing the weight, fuel consumption, and maintenance costs over traditional wired avionics systems. This survey starts with an overview of the major benefits and opportunities in the deployment of wireless technologies for critical applications of an aircraft. The current state-of-art is presented in terms of system classifications based on data rate demands and transceiver installation locations. We then discuss major technical challenges in the design and realization of the envisioned aircraft applications. Although wireless avionics intra-communication has aspects and requirements similar to mission-critical applications of industrial automation, it also has specific issues such as complex structures, operations, and safety of the aircraft that make this area of research self-standing and challenging. To support the critical operations of an aircraft, existing wireless standards for mission-critical industrial applications are briefly discussed to investigate the applicability of the current solutions. Specifically, IEEE 802.15.4-based protocols and Bluetooth are discussed for low data rate applications, whereas IEEE 802.11- based standards are considered for high data rate applications. Eventually, we propose fundamental schemes in terms of network architecture, protocol, and resource management to support the critical avionics applications and discuss the research directions in this emerging area.
△ Less
Submitted 22 June, 2020;
originally announced June 2020.
-
A Determination Scheme for Quasi-Identifiers Using Uniqueness and Influence for De-Identification of Clinical Data
Authors:
Jipmin Jung,
Phillip Park,
Jaedong Lee,
Hyein Lee,
Geonkook Lee,
Hyosoung Cha
Abstract:
Objectives; The accumulation and usefulness of clinical data have increased with IT development. While using clinical data that needs to be identifiable to obtain meaningful information, it is essential to ensure that data is de-identified and unnecessary clinical information is minimized to protect personal information. This process requires criteria and an appropriate method as there are clear i…
▽ More
Objectives; The accumulation and usefulness of clinical data have increased with IT development. While using clinical data that needs to be identifiable to obtain meaningful information, it is essential to ensure that data is de-identified and unnecessary clinical information is minimized to protect personal information. This process requires criteria and an appropriate method as there are clear identifiers as well as quasi-identifiers that are not readily identifiable. Methods; To formulate such a method, first, primary quasi-identifiers were selected by classifying information in 20 clinical personal information database tables into Direct-Identifier (DID), Quasi-Identifier (QI), Sensitive Attribute (SA), and Non-Sensitive Attribute (NSA) according to its type. Secondary QIs were then selected by assessing the risk for outliers by measuring uniqueness values of the selected data and scoring re-identification by calculating equivalence class of the influence on other data on QI removal. Third, the risk of re-identification of data users was numeralized and classified. Lastly, the final QI according to user class was determined by comparing the calculated re-identification scores to the threshold values of user classes. Results; Eventually, final QIs ranging from a minimum of 18 to a maximum of 28 were selected by making an assumption about user classes and using it as criteria. Conclusions; The QI selection method presented by the current investigators can be used by researchers at the final checkup stage before they de-identify the selected QIs. Therefore, clinical data users can securely and efficiently use clinical data containing personal information by objectively selecting QIs using the method proposed in the present study.
△ Less
Submitted 12 April, 2018;
originally announced April 2018.
-
Wireless Network Design for Control Systems: A Survey
Authors:
Pangun Park,
Sinem Coleri Ergen,
Carlo Fischione,
Chenyang Lu,
Karl Henrik Johansson
Abstract:
Wireless networked control systems (WNCS) are composed of spatially distributed sensors, actuators, and con- trollers communicating through wireless networks instead of conventional point-to-point wired connections. Due to their main benefits in the reduction of deployment and maintenance costs, large flexibility and possible enhancement of safety, WNCS are becoming a fundamental infrastructure te…
▽ More
Wireless networked control systems (WNCS) are composed of spatially distributed sensors, actuators, and con- trollers communicating through wireless networks instead of conventional point-to-point wired connections. Due to their main benefits in the reduction of deployment and maintenance costs, large flexibility and possible enhancement of safety, WNCS are becoming a fundamental infrastructure technology for critical control systems in automotive electrical systems, avionics control systems, building management systems, and industrial automation systems. The main challenge in WNCS is to jointly design the communication and control systems considering their tight interaction to improve the control performance and the network lifetime. In this survey, we make an exhaustive review of the literature on wireless network design and optimization for WNCS. First, we discuss what we call the critical interactive variables including sampling period, message delay, message dropout, and network energy consumption. The mutual effects of these communication and control variables motivate their joint tuning. We discuss the effect of controllable wireless network parameters at all layers of the communication protocols on the probability distribution of these interactive variables. We also review the current wireless network standardization for WNCS and their corresponding methodology for adapting the network parameters. Moreover, we discuss the analysis and design of control systems taking into account the effect of the interactive variables on the control system performance. Finally, we present the state-of-the-art wireless network design and optimization for WNCS, while highlighting the tradeoff between the achievable performance and complexity of various approaches. We conclude the survey by highlighting major research issues and identifying future research directions.
△ Less
Submitted 24 August, 2017;
originally announced August 2017.
-
The Opacity Problem in Social Contagion
Authors:
George Berry,
Christopher J. Cameron,
Patrick Park,
Michael W. Macy
Abstract:
Fads, product adoption, mobs, rumors, memes, and emergent norms are diverse social contagions that have been modeled as network cascades. Empirical study of these cascades is vulnerable to what we describe as the "opacity problem": the inability to observe the critical level of peer influence required to trigger an individual's behavioral change. Even with maximal information, network cascades rev…
▽ More
Fads, product adoption, mobs, rumors, memes, and emergent norms are diverse social contagions that have been modeled as network cascades. Empirical study of these cascades is vulnerable to what we describe as the "opacity problem": the inability to observe the critical level of peer influence required to trigger an individual's behavioral change. Even with maximal information, network cascades reveal intervals that bound critical levels of peer exposure, rather than critical values themselves. Existing practice uses interval maxima, which systematically over-estimates the social influence required for behavioral change. Simulations reveal that the over-estimation is likely common and large in magnitude. This is confirmed by an empirical study of hashtag cascades among 3.2 million Twitter users: one in five hashtag adoptions suffers critical value uncertainty due to the opacity problem. Different assumptions about these intervals lead to qualitatively different conclusions about the role of peer reinforcement in diffusion. We introduce a solution that combines identifying tightly bounded intervals with predicting uncertain critical values using node-level information.
△ Less
Submitted 19 November, 2018; v1 submitted 8 February, 2017;
originally announced February 2017.
-
The Mesh of Civilizations and International Email Flows
Authors:
Bogdan State,
Patrick Park,
Ingmar Weber,
Yelena Mejova,
Michael Macy
Abstract:
In The Clash of Civilizations, Samuel Huntington argued that the primary axis of global conflict was no longer ideological or economic but cultural and religious, and that this division would characterize the "battle lines of the future." In contrast to the "top down" approach in previous research focused on the relations among nation states, we focused on the flows of interpersonal communication…
▽ More
In The Clash of Civilizations, Samuel Huntington argued that the primary axis of global conflict was no longer ideological or economic but cultural and religious, and that this division would characterize the "battle lines of the future." In contrast to the "top down" approach in previous research focused on the relations among nation states, we focused on the flows of interpersonal communication as a bottom-up view of international alignments. To that end, we mapped the locations of the world's countries in global email networks to see if we could detect cultural fault lines. Using IP-geolocation on a worldwide anonymized dataset obtained from a large Internet company, we constructed a global email network. In computing email flows we employ a novel rescaling procedure to account for differences due to uneven adoption of a particular Internet service across the world. Our analysis shows that email flows are consistent with Huntington's thesis. In addition to location in Huntington's "civilizations," our results also attest to the importance of both cultural and economic factors in the patterning of inter-country communication ties.
△ Less
Submitted 10 March, 2013; v1 submitted 28 February, 2013;
originally announced March 2013.
