-
Smell of Fire Increases Behavioural Realism in Virtual Reality: A Case Study on a Recreated MGM Grand Hotel Fire
Authors:
Humayun Khan,
Daniel Nilsson
Abstract:
Virtual reality allows creating highly immersive visual and auditory experiences, making users feel physically present in the environment. This makes it an ideal platform to simulate dangerous scenarios, including fire evacuation, and study human behaviour without exposing users to harmful elements. However, human perception of the surroundings is based on the integration of multiple sensory cues…
▽ More
Virtual reality allows creating highly immersive visual and auditory experiences, making users feel physically present in the environment. This makes it an ideal platform to simulate dangerous scenarios, including fire evacuation, and study human behaviour without exposing users to harmful elements. However, human perception of the surroundings is based on the integration of multiple sensory cues (visual, auditory, tactile, or/and olfactory) present in the environment. When some of the sensory stimuli are missing in the virtual experience, it can break the illusion of being there in the environment and could lead to actions that deviate from normal behaviour. In this work, we added an olfactory cue in a well-documented historic hotel fire scenario that was recreated in VR, and examined the effects of the olfactory cue on human behaviour. We conducted a between subject study on 40 naive participants. Our results show that the addition of the olfactory cue could increase behavioural realism. We found that 80% of the studied actions for the VR with olfactory cue condition matched the ones performed by the survivors. In comparison, only 40% of the participants' actions for VR only condition were similar to the survivors.
△ Less
Submitted 13 November, 2023;
originally announced November 2023.
-
FlowIBR: Leveraging Pre-Training for Efficient Neural Image-Based Rendering of Dynamic Scenes
Authors:
Marcel Büsching,
Josef Bengtson,
David Nilsson,
Mårten Björkman
Abstract:
We introduce FlowIBR, a novel approach for efficient monocular novel view synthesis of dynamic scenes. Existing techniques already show impressive rendering quality but tend to focus on optimization within a single scene without leveraging prior knowledge, resulting in long optimization times per scene. FlowIBR circumvents this limitation by integrating a neural image-based rendering method, pre-t…
▽ More
We introduce FlowIBR, a novel approach for efficient monocular novel view synthesis of dynamic scenes. Existing techniques already show impressive rendering quality but tend to focus on optimization within a single scene without leveraging prior knowledge, resulting in long optimization times per scene. FlowIBR circumvents this limitation by integrating a neural image-based rendering method, pre-trained on a large corpus of widely available static scenes, with a per-scene optimized scene flow field. Utilizing this flow field, we bend the camera rays to counteract the scene dynamics, thereby presenting the dynamic scene as if it were static to the rendering network. The proposed method reduces per-scene optimization time by an order of magnitude, achieving comparable rendering quality to existing methods -- all on a single consumer-grade GPU.
△ Less
Submitted 15 April, 2024; v1 submitted 11 September, 2023;
originally announced September 2023.
-
Adjustable Visual Appearance for Generalizable Novel View Synthesis
Authors:
Josef Bengtson,
David Nilsson,
Che-Tsung Lin,
Marcel Büsching,
Fredrik Kahl
Abstract:
We present a generalizable novel view synthesis method which enables modifying the visual appearance of an observed scene so rendered views match a target weather or lighting condition without any scene specific training or access to reference views at the target condition. Our method is based on a pretrained generalizable transformer architecture and is fine-tuned on synthetically generated scene…
▽ More
We present a generalizable novel view synthesis method which enables modifying the visual appearance of an observed scene so rendered views match a target weather or lighting condition without any scene specific training or access to reference views at the target condition. Our method is based on a pretrained generalizable transformer architecture and is fine-tuned on synthetically generated scenes under different appearance conditions. This allows for rendering novel views in a consistent manner for 3D scenes that were not included in the training set, along with the ability to (i) modify their appearance to match the target condition and (ii) smoothly interpolate between different conditions. Experiments on real and synthetic scenes show that our method is able to generate 3D consistent renderings while making realistic appearance changes, including qualitative and quantitative comparisons. Please refer to our project page for video results: https://ava-nvs.github.io/
△ Less
Submitted 26 January, 2024; v1 submitted 2 June, 2023;
originally announced June 2023.
-
Situational-Aware Multi-Graph Convolutional Recurrent Network (SA-MGCRN) for Travel Demand Forecasting During Wildfires
Authors:
Xiaojian Zhang,
Xilei Zhao,
Yiming Xu,
Ruggiero Lovreglio,
Daniel Nilsson
Abstract:
Real-time forecasting of travel demand during wildfire evacuations is crucial for emergency managers and transportation planners to make timely and better-informed decisions. However, few studies focus on accurate travel demand forecasting in large-scale emergency evacuations. Therefore, this study develops and tests a new methodological framework for modeling trip generation in wildfire evacuatio…
▽ More
Real-time forecasting of travel demand during wildfire evacuations is crucial for emergency managers and transportation planners to make timely and better-informed decisions. However, few studies focus on accurate travel demand forecasting in large-scale emergency evacuations. Therefore, this study develops and tests a new methodological framework for modeling trip generation in wildfire evacuations by using (a) large-scale GPS data generated by mobile devices and (b) state-of-the-art AI technologies. The proposed methodology aims at forecasting evacuation trips and other types of trips. Based on the travel demand inferred from the GPS data, we develop a new deep learning model, i.e., Situational-Aware Multi-Graph Convolutional Recurrent Network (SA-MGCRN), along with a model updating scheme to achieve real-time forecasting of travel demand during wildfire evacuations. The proposed methodological framework is tested in this study for a real-world case study: the 2019 Kincade Fire in Sonoma County, CA. The results show that SA-MGCRN significantly outperforms all the selected state-of-the-art benchmarks in terms of prediction performance. Our finding suggests that the most important model components of SA-MGCRN are evacuation order/warning information, proximity to fire, and population change, which are consistent with behavioral theories and empirical findings.
△ Less
Submitted 12 April, 2023;
originally announced April 2023.
-
Embodied Learning for Lifelong Visual Perception
Authors:
David Nilsson,
Aleksis Pirinen,
Erik Gärtner,
Cristian Sminchisescu
Abstract:
We study lifelong visual perception in an embodied setup, where we develop new models and compare various agents that navigate in buildings and occasionally request annotations which, in turn, are used to refine their visual perception capabilities. The purpose of the agents is to recognize objects and other semantic classes in the whole building at the end of a process that combines exploration a…
▽ More
We study lifelong visual perception in an embodied setup, where we develop new models and compare various agents that navigate in buildings and occasionally request annotations which, in turn, are used to refine their visual perception capabilities. The purpose of the agents is to recognize objects and other semantic classes in the whole building at the end of a process that combines exploration and active visual learning. As we study this task in a lifelong learning context, the agents should use knowledge gained in earlier visited environments in order to guide their exploration and active learning strategy in successively visited buildings. We use the semantic segmentation performance as a proxy for general visual perception and study this novel task for several exploration and annotation methods, ranging from frontier exploration baselines which use heuristic active learning, to a fully learnable approach. For the latter, we introduce a deep reinforcement learning (RL) based agent which jointly learns both navigation and active learning. A point goal navigation formulation, coupled with a global planner which supplies goals, is integrated into the RL model in order to provide further incentives for systematic exploration of novel scenes. By performing extensive experiments on the Matterport3D dataset, we show how the proposed agents can utilize knowledge from previously explored scenes when exploring new ones, e.g. through less granular exploration and less frequent requests for annotations. The results also suggest that a learning-based agent is able to use its prior visual knowledge more effectively than heuristic alternatives.
△ Less
Submitted 28 December, 2021;
originally announced December 2021.
-
Estimating Wildfire Evacuation Decision and Departure Timing Using Large-Scale GPS Data
Authors:
Xilei Zhao,
Yiming Xu,
Ruggiero Lovreglio,
Erica Kuligowski,
Daniel Nilsson,
Thomas Cova,
Alex Wu,
Xiang Yan
Abstract:
With increased frequency and intensity due to climate change, wildfires have become a growing global concern. This creates severe challenges for fire and emergency services as well as communities in the wildland-urban interface (WUI). To reduce wildfire risk and enhance the safety of WUI communities, improving our understanding of wildfire evacuation is a pressing need. To this end, this study pro…
▽ More
With increased frequency and intensity due to climate change, wildfires have become a growing global concern. This creates severe challenges for fire and emergency services as well as communities in the wildland-urban interface (WUI). To reduce wildfire risk and enhance the safety of WUI communities, improving our understanding of wildfire evacuation is a pressing need. To this end, this study proposes a new methodology to analyze human behavior during wildfires by leveraging a large-scale GPS dataset. This methodology includes a home-location inference algorithm and an evacuation-behavior inference algorithm, to systematically identify different groups of wildfire evacuees (i.e., self-evacuee, shadow evacuee, evacuee under warning, and ordered evacuee). We applied the methodology to the 2019 Kincade Fire in Sonoma County, CA. We found that among all groups of evacuees, self-evacuees and shadow evacuees accounted for more than half of the evacuees during the Kincade Fire. The results also show that inside of the evacuation warning/order zones, the total evacuation compliance rate was around 46% among all the categorized people. The findings of this study can be used by emergency managers and planners to better target public outreach campaigns, training protocols, and emergency communication strategies to prepare WUI households for future wildfire events.
△ Less
Submitted 16 September, 2021;
originally announced September 2021.
-
Embodied Visual Active Learning for Semantic Segmentation
Authors:
David Nilsson,
Aleksis Pirinen,
Erik Gärtner,
Cristian Sminchisescu
Abstract:
We study the task of embodied visual active learning, where an agent is set to explore a 3d environment with the goal to acquire visual scene understanding by actively selecting views for which to request annotation. While accurate on some benchmarks, today's deep visual recognition pipelines tend to not generalize well in certain real-world scenarios, or for unusual viewpoints. Robotic perception…
▽ More
We study the task of embodied visual active learning, where an agent is set to explore a 3d environment with the goal to acquire visual scene understanding by actively selecting views for which to request annotation. While accurate on some benchmarks, today's deep visual recognition pipelines tend to not generalize well in certain real-world scenarios, or for unusual viewpoints. Robotic perception, in turn, requires the capability to refine the recognition capabilities for the conditions where the mobile system operates, including cluttered indoor environments or poor illumination. This motivates the proposed task, where an agent is placed in a novel environment with the objective of improving its visual recognition capability. To study embodied visual active learning, we develop a battery of agents - both learnt and pre-specified - and with different levels of knowledge of the environment. The agents are equipped with a semantic segmentation network and seek to acquire informative views, move and explore in order to propagate annotations in the neighbourhood of those views, then refine the underlying segmentation network by online retraining. The trainable method uses deep reinforcement learning with a reward function that balances two competing objectives: task performance, represented as visual recognition accuracy, which requires exploring the environment, and the necessary amount of annotated data requested during active exploration. We extensively evaluate the proposed models using the photorealistic Matterport3D simulator and show that a fully learnt method outperforms comparable pre-specified counterparts, even when requesting fewer annotations.
△ Less
Submitted 17 December, 2020;
originally announced December 2020.
-
Semantic Video Segmentation by Gated Recurrent Flow Propagation
Authors:
David Nilsson,
Cristian Sminchisescu
Abstract:
Semantic video segmentation is challenging due to the sheer amount of data that needs to be processed and labeled in order to construct accurate models. In this paper we present a deep, end-to-end trainable methodology to video segmentation that is capable of leveraging information present in unlabeled data in order to improve semantic estimates. Our model combines a convolutional architecture and…
▽ More
Semantic video segmentation is challenging due to the sheer amount of data that needs to be processed and labeled in order to construct accurate models. In this paper we present a deep, end-to-end trainable methodology to video segmentation that is capable of leveraging information present in unlabeled data in order to improve semantic estimates. Our model combines a convolutional architecture and a spatio-temporal transformer recurrent layer that are able to temporally propagate labeling information by means of optical flow, adaptively gated based on its locally estimated uncertainty. The flow, the recognition and the gated temporal propagation modules can be trained jointly, end-to-end. The temporal, gated recurrent flow propagation component of our model can be plugged into any static semantic segmentation architecture and turn it into a weakly supervised video processing one. Our extensive experiments in the challenging CityScapes and Camvid datasets, and based on multiple deep architectures, indicate that the resulting model can leverage unlabeled temporal frames, next to a labeled one, in order to improve both the video segmentation accuracy and the consistency of its temporal labeling, at no additional annotation cost and with little extra computation.
△ Less
Submitted 2 October, 2017; v1 submitted 28 December, 2016;
originally announced December 2016.
-
Evaluating Influence Diagrams using LIMIDs
Authors:
Dennis Nilsson,
Steffen L. Lauritzen
Abstract:
We present a new approach to the solution of decision problems formulated as influence diagrams. The approach converts the influence diagram into a simpler structure, the LImited Memory Influence Diagram (LIMID), where only the requisite information for the computation of optimal policies is depicted. Because the requisite information is explicitly represented in the diagram, the evaluation proced…
▽ More
We present a new approach to the solution of decision problems formulated as influence diagrams. The approach converts the influence diagram into a simpler structure, the LImited Memory Influence Diagram (LIMID), where only the requisite information for the computation of optimal policies is depicted. Because the requisite information is explicitly represented in the diagram, the evaluation procedure can take advantage of it. In this paper we show how to convert an influence diagram to a LIMID and describe the procedure for finding an optimal strategy. Our approach can yield significant savings of memory and computational time when compared to traditional methods.
△ Less
Submitted 16 January, 2013;
originally announced January 2013.
-
Solving Influence Diagrams using HUGIN, Shafer-Shenoy and Lazy Propagation
Authors:
Anders L. Madsen,
Dennis Nilsson
Abstract:
In this paper we compare three different architectures for the evaluation of influence diagrams: HUGIN, Shafer-Shenoy, and Lazy Evaluation architecture. The computational complexity of the architectures are compared on the LImited Memory Influence Diagram (LIMID): a diagram where only the requiste information for the computation of the optimal policies are depicted. Because the requsite informatio…
▽ More
In this paper we compare three different architectures for the evaluation of influence diagrams: HUGIN, Shafer-Shenoy, and Lazy Evaluation architecture. The computational complexity of the architectures are compared on the LImited Memory Influence Diagram (LIMID): a diagram where only the requiste information for the computation of the optimal policies are depicted. Because the requsite information is explicitly represented in the LIMID the evaluation can take advantage of it, and significant savings in computational can be obtained. In this paper we show how the obtained savings is considerably increased when the computations performed on the LIMID is according to the Lazy Evaluation scheme.
△ Less
Submitted 10 January, 2013;
originally announced January 2013.
