-
Harms in Repurposing Real-World Sensory Cues for Mixed Reality: A Causal Perspective
Authors:
Yujie Tao,
Sean Follmer
Abstract:
The rise of Mixed Reality (MR) stimulates new interactive techniques that seamlessly blend the virtual and physical environments. Just as virtual content could be overlayed onto the physical world for providing adaptive user interfaces [5, 8], emergent techniques "repurpose" everyday environments and sensory cues to support the virtual content [7, 9, 13-15]. For instance, a strong wind gust in the…
▽ More
The rise of Mixed Reality (MR) stimulates new interactive techniques that seamlessly blend the virtual and physical environments. Just as virtual content could be overlayed onto the physical world for providing adaptive user interfaces [5, 8], emergent techniques "repurpose" everyday environments and sensory cues to support the virtual content [7, 9, 13-15]. For instance, a strong wind gust in the real world, rather than being distracting to the virtual experience, can be mapped with trees swaying in MR to achieve a unifying experience [15], as shown in Figure 1. Such techniques introduce stronger immersion, but they also expose users to overlooked perceptual manipulations, where safety risks arise from misperception of real-world events. In this work, we apply a causal inference perspective to understand the harms of repurposing real-world sensory cues for MR. We argue that by viewing the MR experience as a causal inference process of interpreting cues arising from both the virtual and physical world, MR designers and researchers can gain a new lens to understand potential perceptual manipulation harms.
△ Less
Submitted 23 April, 2024;
originally announced May 2024.
-
Electroadhesive Auxetics as Programmable Layer Jamming Skins for Formable Crust Shape Displays
Authors:
Ahad M. Rauf,
Jack S. Bernardo,
Sean Follmer
Abstract:
Shape displays are a class of haptic devices that enable whole-hand haptic exploration of 3D surfaces. However, their scalability is limited by the mechanical complexity and high cost of traditional actuator arrays. In this paper, we propose using electroadhesive auxetic skins as a strain-limiting layer to create programmable shape change in a continuous ("formable crust") shape display. Auxetic s…
▽ More
Shape displays are a class of haptic devices that enable whole-hand haptic exploration of 3D surfaces. However, their scalability is limited by the mechanical complexity and high cost of traditional actuator arrays. In this paper, we propose using electroadhesive auxetic skins as a strain-limiting layer to create programmable shape change in a continuous ("formable crust") shape display. Auxetic skins are manufactured as flexible printed circuit boards with dielectric-laminated electrodes on each auxetic unit cell (AUC), using monolithic fabrication to lower cost and assembly time. By layering multiple sheets and applying a voltage between electrodes on subsequent layers, electroadhesion locks individual AUCs, achieving a maximum in-plane stiffness variation of 7.6x with a power consumption of 50 uW/AUC. We first characterize an individual AUC and compare results to a kinematic model. We then validate the ability of a 5x5 AUC array to actively modify its own axial and transverse stiffness. Finally, we demonstrate this array in a continuous shape display as a strain-limiting skin to programmatically modulate the shape output of an inflatable LDPE pouch. Integrating electroadhesion with auxetics enables new capabilities for scalable, low-profile, and low-power control of flexible robotic systems.
△ Less
Submitted 11 March, 2023; v1 submitted 10 November, 2022;
originally announced November 2022.
-
Beyond Being Real: A Sensorimotor Control Perspective on Interactions in Virtual Reality
Authors:
Parastoo Abtahi,
Sidney Q. Hough,
James A. Landay,
Sean Follmer
Abstract:
We can create Virtual Reality (VR) interactions that have no equivalent in the real world by remapping spacetime or altering users' body representation, such as stretching the user's virtual arm for manipulation of distant objects or scaling up the user's avatar to enable rapid locomotion. Prior research has leveraged such approaches, what we call beyond-real techniques, to make interactions in VR…
▽ More
We can create Virtual Reality (VR) interactions that have no equivalent in the real world by remapping spacetime or altering users' body representation, such as stretching the user's virtual arm for manipulation of distant objects or scaling up the user's avatar to enable rapid locomotion. Prior research has leveraged such approaches, what we call beyond-real techniques, to make interactions in VR more practical, efficient, ergonomic, and accessible. We present a survey categorizing prior movement-based VR interaction literature as reality-based, illusory, or beyond-real interactions. We survey relevant conferences (CHI, IEEE VR, VRST, UIST, and DIS) while focusing on selection, manipulation, locomotion, and navigation in VR. For beyond-real interactions, we describe the transformations that have been used by prior works to create novel remappings. We discuss open research questions through the lens of the human sensorimotor control system and highlight challenges that need to be addressed for effective utilization of beyond-real interactions in future VR applications, including plausibility, control, long-term adaptation, and individual differences.
△ Less
Submitted 18 April, 2022;
originally announced April 2022.
-
Social-Cultural Factors in the Design of Technology for Hispanic People with Stroke
Authors:
Elizabeth D. Vasquez,
Allison M. Okamura,
Sean Follmer
Abstract:
Stroke is a leading cause of serious, long-term disability in the United States. There exist disparities in both stroke prevalence and outcomes between people with stroke in Hispanic and Latinx communities and the general stroke population. Current stroke technology - which aims to improve quality of life and bring people with stroke to the most functional, independent state possible - has shown p…
▽ More
Stroke is a leading cause of serious, long-term disability in the United States. There exist disparities in both stroke prevalence and outcomes between people with stroke in Hispanic and Latinx communities and the general stroke population. Current stroke technology - which aims to improve quality of life and bring people with stroke to the most functional, independent state possible - has shown promising results for the general stroke population, but has failed to close the recovery outcome gap for underserved Hispanic and Latinx people with stroke. Previous work in health education, digital health, and HRI has improved human health outcomes by incorporating social-cultural factors, though not for stroke. In this position paper, we aim to justify accounting for unique cultural factors in stroke technology design for the Hispanic and Latinx community. We review examples of successful culturally appropriate interventions and suggest design considerations (mutually beneficial community consultation, accommodating for barriers beforehand, building on culture, and incorporating education of the family) to provide more culturally appropriate design of Hispanic and Latinx stroke technology and reduce the disparity gap.
△ Less
Submitted 16 March, 2022;
originally announced March 2022.
-
Acoustic Communication and Sensing for Inflatable Modular Soft Robots
Authors:
D. S. Drew,
M. Devlin,
E. Hawkes,
S. Follmer
Abstract:
Modular soft robots combine the strengths of two traditionally separate areas of robotics. As modular robots, they can show robustness to individual failure and reconfigurability; as soft robots, they can deform and undergo large shape changes in order to adapt to their environment, and have inherent human safety. However, for sensing and communication these robots also combine the challenges of b…
▽ More
Modular soft robots combine the strengths of two traditionally separate areas of robotics. As modular robots, they can show robustness to individual failure and reconfigurability; as soft robots, they can deform and undergo large shape changes in order to adapt to their environment, and have inherent human safety. However, for sensing and communication these robots also combine the challenges of both: they require solutions that are scalable (low cost and complexity) and efficient (low power) to enable collectives of large numbers of robots, and these solutions must also be able to interface with the high extension ratio elastic bodies of soft robots. In this work, we seek to address these challenges using acoustic signals produced by piezoelectric surface transducers that are cheap, simple, and low power, and that not only integrate with but also leverage the elastic robot skins for signal transmission. Importantly, to further increase scalability, the transducers exhibit multi-functionality made possible by a relatively flat frequency response across the audible and ultrasonic ranges. With minimal hardware, they enable directional contact-based communication, audible-range communication at a distance, and exteroceptive sensing. We demonstrate a subset of the decentralized collective behaviors these functions make possible with multi-robot hardware implementations. The use of acoustic waves in this domain is shown to provide distinct advantages over existing solutions.
△ Less
Submitted 27 January, 2021;
originally announced January 2021.
-
Interaction with Ubiquitous Robots and Autonomous IoT
Authors:
Lawrence H. Kim,
Sean Follmer
Abstract:
Robotics have been slowly permeating Internet of Things (IoT) where the previously ubiquitous but static sensors are now given the power to actively navigate the environment and even interact with users. Emergence of these ubiquitous swarms of robots not only opens up the range of possible applications, but also increases the number of elements to study and design for. We do not yet understand how…
▽ More
Robotics have been slowly permeating Internet of Things (IoT) where the previously ubiquitous but static sensors are now given the power to actively navigate the environment and even interact with users. Emergence of these ubiquitous swarms of robots not only opens up the range of possible applications, but also increases the number of elements to study and design for. We do not yet understand how, when, and where these robots should move, manipulate, and touch around people. Through user-centered studies, we aim to better understand how to best design for interaction with Autonomous IoT or a swarm of ubiquitous robots.
△ Less
Submitted 17 June, 2019;
originally announced November 2019.
