-
Adapting to Covariate Shift in Real-time by Encoding Trees with Motion Equations
Authors:
Tham Yik Foong,
Heng Zhang,
Mao Po Yuan,
Danilo Vasconcellos Vargas
Abstract:
Input distribution shift presents a significant problem in many real-world systems. Here we present Xenovert, an adaptive algorithm that can dynamically adapt to changes in input distribution. It is a perfect binary tree that adaptively divides a continuous input space into several intervals of uniform density while receiving a continuous stream of input. This process indirectly maps the source di…
▽ More
Input distribution shift presents a significant problem in many real-world systems. Here we present Xenovert, an adaptive algorithm that can dynamically adapt to changes in input distribution. It is a perfect binary tree that adaptively divides a continuous input space into several intervals of uniform density while receiving a continuous stream of input. This process indirectly maps the source distribution to the shifted target distribution, preserving the data's relationship with the downstream decoder/operation, even after the shift occurs. In this paper, we demonstrated how a neural network integrated with Xenovert achieved better results in 4 out of 5 shifted datasets, saving the hurdle of retraining a machine learning model. We anticipate that Xenovert can be applied to many more applications that require adaptation to unforeseen input distribution shifts, even when the distribution shift is drastic.
△ Less
Submitted 7 April, 2024;
originally announced April 2024.
-
The POLAR Traverse Dataset: A Dataset of Stereo Camera Images Simulating Traverses across Lunar Polar Terrain under Extreme Lighting Conditions
Authors:
Margaret Hansen,
Uland Wong,
Terrence Fong
Abstract:
We present the POLAR Traverse Dataset: a dataset of high-fidelity stereo pair images of lunar-like terrain under polar lighting conditions designed to simulate a straight-line traverse. Images from individual traverses with different camera heights and pitches were recorded at 1 m intervals by moving a suspended stereo bar across a test bed filled with regolith simulant and shaped to mimic lunar s…
▽ More
We present the POLAR Traverse Dataset: a dataset of high-fidelity stereo pair images of lunar-like terrain under polar lighting conditions designed to simulate a straight-line traverse. Images from individual traverses with different camera heights and pitches were recorded at 1 m intervals by moving a suspended stereo bar across a test bed filled with regolith simulant and shaped to mimic lunar south polar terrain. Ground truth geometry and camera position information was also recorded. This dataset is intended for developing and testing software algorithms that rely on stereo or monocular camera images, such as visual odometry, for use in the lunar polar environment, as well as to provide insight into the expected lighting conditions in lunar polar regions.
△ Less
Submitted 18 March, 2024;
originally announced March 2024.
-
Large Language Model Capabilities in Perioperative Risk Prediction and Prognostication
Authors:
Philip Chung,
Christine T Fong,
Andrew M Walters,
Nima Aghaeepour,
Meliha Yetisgen,
Vikas N O'Reilly-Shah
Abstract:
We investigate whether general-domain large language models such as GPT-4 Turbo can perform risk stratification and predict post-operative outcome measures using a description of the procedure and a patient's clinical notes derived from the electronic health record. We examine predictive performance on 8 different tasks: prediction of ASA Physical Status Classification, hospital admission, ICU adm…
▽ More
We investigate whether general-domain large language models such as GPT-4 Turbo can perform risk stratification and predict post-operative outcome measures using a description of the procedure and a patient's clinical notes derived from the electronic health record. We examine predictive performance on 8 different tasks: prediction of ASA Physical Status Classification, hospital admission, ICU admission, unplanned admission, hospital mortality, PACU Phase 1 duration, hospital duration, and ICU duration. Few-shot and chain-of-thought prompting improves predictive performance for several of the tasks. We achieve F1 scores of 0.50 for ASA Physical Status Classification, 0.81 for ICU admission, and 0.86 for hospital mortality. Performance on duration prediction tasks were universally poor across all prompt strategies. Current generation large language models can assist clinicians in perioperative risk stratification on classification tasks and produce high-quality natural language summaries and explanations.
△ Less
Submitted 3 January, 2024;
originally announced January 2024.
-
Synthetic Shifts to Initial Seed Vector Exposes the Brittle Nature of Latent-Based Diffusion Models
Authors:
Mao Po-Yuan,
Shashank Kotyan,
Tham Yik Foong,
Danilo Vasconcellos Vargas
Abstract:
Recent advances in Conditional Diffusion Models have led to substantial capabilities in various domains. However, understanding the impact of variations in the initial seed vector remains an underexplored area of concern. Particularly, latent-based diffusion models display inconsistencies in image generation under standard conditions when initialized with suboptimal initial seed vectors. To unders…
▽ More
Recent advances in Conditional Diffusion Models have led to substantial capabilities in various domains. However, understanding the impact of variations in the initial seed vector remains an underexplored area of concern. Particularly, latent-based diffusion models display inconsistencies in image generation under standard conditions when initialized with suboptimal initial seed vectors. To understand the impact of the initial seed vector on generated samples, we propose a reliability evaluation framework that evaluates the generated samples of a diffusion model when the initial seed vector is subjected to various synthetic shifts. Our results indicate that slight manipulations to the initial seed vector of the state-of-the-art Stable Diffusion (Rombach et al., 2022) can lead to significant disturbances in the generated samples, consequently creating images without the effect of conditioning variables. In contrast, GLIDE (Nichol et al., 2022) stands out in generating reliable samples even when the initial seed vector is transformed. Thus, our study sheds light on the importance of the selection and the impact of the initial seed vector in the latent-based diffusion model.
△ Less
Submitted 24 November, 2023;
originally announced December 2023.
-
The Challenges of Image Generation Models in Generating Multi-Component Images
Authors:
Tham Yik Foong,
Shashank Kotyan,
Po Yuan Mao,
Danilo Vasconcellos Vargas
Abstract:
Recent advances in text-to-image generators have led to substantial capabilities in image generation. However, the complexity of prompts acts as a bottleneck in the quality of images generated. A particular under-explored facet is the ability of generative models to create high-quality images comprising multiple components given as a prior. In this paper, we propose and validate a metric called Co…
▽ More
Recent advances in text-to-image generators have led to substantial capabilities in image generation. However, the complexity of prompts acts as a bottleneck in the quality of images generated. A particular under-explored facet is the ability of generative models to create high-quality images comprising multiple components given as a prior. In this paper, we propose and validate a metric called Components Inclusion Score (CIS) to evaluate the extent to which a model can correctly generate multiple components. Our results reveal that the evaluated models struggle to incorporate all the visual elements from prompts with multiple components (8.53% drop in CIS per component for all evaluated models). We also identify a significant decline in the quality of the images and context awareness within an image as the number of components increased (15.91% decrease in inception Score and 9.62% increase in Frechet Inception Distance). To remedy this issue, we fine-tuned Stable Diffusion V2 on a custom-created test dataset with multiple components, outperforming its vanilla counterpart. To conclude, these findings reveal a critical limitation in existing text-to-image generators, shedding light on the challenge of generating multiple components within a single image using a complex prompt.
△ Less
Submitted 22 November, 2023;
originally announced November 2023.
-
Generating Oscillation Activity with Echo State Network to Mimic the Behavior of a Simple Central Pattern Generator
Authors:
Tham Yik Foong,
Danilo Vasconcellos Vargas
Abstract:
This paper presents a method for reproducing a simple central pattern generator (CPG) using a modified Echo State Network (ESN). Conventionally, the dynamical reservoir needs to be damped to stabilize and preserve memory. However, we find that a reservoir that develops oscillatory activity without any external excitation can mimic the behaviour of a simple CPG in biological systems. We define the…
▽ More
This paper presents a method for reproducing a simple central pattern generator (CPG) using a modified Echo State Network (ESN). Conventionally, the dynamical reservoir needs to be damped to stabilize and preserve memory. However, we find that a reservoir that develops oscillatory activity without any external excitation can mimic the behaviour of a simple CPG in biological systems. We define the specific neuron ensemble required for generating oscillations in the reservoir and demonstrate how adjustments to the leaking rate, spectral radius, topology, and population size can increase the probability of reproducing these oscillations. The results of the experiments, conducted on the time series simulation tasks, demonstrate that the ESN is able to generate the desired waveform without any input. This approach offers a promising solution for the development of bio-inspired controllers for robotic systems.
△ Less
Submitted 19 June, 2023;
originally announced June 2023.
-
Dynamical Equations With Bottom-up Self-Organizing Properties Learn Accurate Dynamical Hierarchies Without Any Loss Function
Authors:
Danilo Vasconcellos Vargas,
Tham Yik Foong,
Heng Zhang
Abstract:
Self-organization is ubiquitous in nature and mind. However, machine learning and theories of cognition still barely touch the subject. The hurdle is that general patterns are difficult to define in terms of dynamical equations and designing a system that could learn by reordering itself is still to be seen. Here, we propose a learning system, where patterns are defined within the realm of nonline…
▽ More
Self-organization is ubiquitous in nature and mind. However, machine learning and theories of cognition still barely touch the subject. The hurdle is that general patterns are difficult to define in terms of dynamical equations and designing a system that could learn by reordering itself is still to be seen. Here, we propose a learning system, where patterns are defined within the realm of nonlinear dynamics with positive and negative feedback loops, allowing attractor-repeller pairs to emerge for each pattern observed. Experiments reveal that such a system can map temporal to spatial correlation, enabling hierarchical structures to be learned from sequential data. The results are accurate enough to surpass state-of-the-art unsupervised learning algorithms in seven out of eight experiments as well as two real-world problems. Interestingly, the dynamic nature of the system makes it inherently adaptive, giving rise to phenomena similar to phase transitions in chemistry/thermodynamics when the input structure changes. Thus, the work here sheds light on how self-organization can allow for pattern recognition and hints at how intelligent behavior might emerge from simple dynamic equations without any objective/loss function.
△ Less
Submitted 4 February, 2023;
originally announced February 2023.
-
Registering Articulated Objects With Human-in-the-loop Corrections
Authors:
Michael Hagenow,
Emmanuel Senft,
Evan Laske,
Kimberly Hambuchen,
Terrence Fong,
Robert Radwin,
Michael Gleicher,
Bilge Mutlu,
Michael Zinn
Abstract:
Remotely programming robots to execute tasks often relies on registering objects of interest in the robot's environment. Frequently, these tasks involve articulating objects such as opening or closing a valve. However, existing human-in-the-loop methods for registering objects do not consider articulations and the corresponding impact to the geometry of the object, which can cause the methods to f…
▽ More
Remotely programming robots to execute tasks often relies on registering objects of interest in the robot's environment. Frequently, these tasks involve articulating objects such as opening or closing a valve. However, existing human-in-the-loop methods for registering objects do not consider articulations and the corresponding impact to the geometry of the object, which can cause the methods to fail. In this work, we present an approach where the registration system attempts to automatically determine the object model, pose, and articulation for user-selected points using nonlinear fitting and the iterative closest point algorithm. When the fitting is incorrect, the operator can iteratively intervene with corrections after which the system will refit the object. We present an implementation of our fitting procedure for one degree-of-freedom (DOF) objects with revolute joints and evaluate it with a user study that shows that it can improve user performance, in measures of time on task and task load, ease of use, and usefulness compared to a manual registration approach. We also present a situated example that integrates our method into an end-to-end system for articulating a remote valve.
△ Less
Submitted 16 August, 2022; v1 submitted 11 March, 2022;
originally announced March 2022.
-
Affordance Template Registration via Human-in-the-loop Corrections
Authors:
Michael Hagenow,
Michael Zinn,
Terrence Fong,
Evan Laske,
Kimberly Hambuchen
Abstract:
Affordance Templates (ATs) are a method for parameterizing objects for autonomous robot manipulations. In this approach, instances of an object are registered by positioning a model in a 3D environment, which requires a large amount of user input. We instead propose a registration method which combines autonomy and user corrections. For selected objects, the system determines both the model and co…
▽ More
Affordance Templates (ATs) are a method for parameterizing objects for autonomous robot manipulations. In this approach, instances of an object are registered by positioning a model in a 3D environment, which requires a large amount of user input. We instead propose a registration method which combines autonomy and user corrections. For selected objects, the system determines both the model and corresponding pose autonomously. The user makes corrections only if the model or pose is incorrect. This method increases the level of autonomy compared to existing approaches which can reduce user input and time on task. In this paper, we present an overview of existing methods, a description of our method, preliminary results, and planned future work.
△ Less
Submitted 28 September, 2021;
originally announced September 2021.
-
Multi-Modal Active Perception for Information Gathering in Science Missions
Authors:
Akash Arora,
P. Michael Furlong,
Robert Fitch,
Salah Sukkarieh,
Terrence Fong
Abstract:
Robotic science missions in remote environments, such as deep ocean and outer space, can involve studying phenomena that cannot directly be observed using on-board sensors but must be deduced by combining measurements of correlated variables with domain knowledge. Traditionally, in such missions, robots passively gather data along prescribed paths, while inference, path planning, and other high le…
▽ More
Robotic science missions in remote environments, such as deep ocean and outer space, can involve studying phenomena that cannot directly be observed using on-board sensors but must be deduced by combining measurements of correlated variables with domain knowledge. Traditionally, in such missions, robots passively gather data along prescribed paths, while inference, path planning, and other high level decision making is largely performed by a supervisory science team. However, communication constraints hinder these processes, and hence the rate of scientific progress. This paper presents an active perception approach that aims to reduce robots' reliance on human supervision and improve science productivity by encoding scientists' domain knowledge and decision making process on-board. We use Bayesian networks to compactly model critical aspects of scientific knowledge while remaining robust to observation and modeling uncertainty. We then formulate path planning and sensor scheduling as an information gain maximization problem, and propose a sampling-based solution based on Monte Carlo tree search to plan informative sensing actions which exploit the knowledge encoded in the network. The computational complexity of our framework does not grow with the number of observations taken and allows long horizon planning in an anytime manner, making it highly applicable to field robotics. Simulation results show statistically significant performance improvements over baseline methods, and we validate the practicality of our approach through both hardware experiments and simulated experiments with field data gathered during the NASA Mojave Volatiles Prospector science expedition.
△ Less
Submitted 27 December, 2017;
originally announced December 2017.
