Showing 1–33 of 33 results for author: Braud, T

GSLoc: Efficient Camera Pose Refinement via 3D Gaussian Splatting

Authors: Changkun Liu, Shuai Chen, Yash Bhalgat, Siyan Hu, Zirui Wang, Ming Cheng, Victor Adrian Prisacariu, Tristan Braud

Abstract: We leverage 3D Gaussian Splatting (3DGS) as a scene representation and propose a novel test-time camera pose refinement framework, GSLoc. This framework enhances the localization accuracy of state-of-the-art absolute pose regression and scene coordinate regression methods. The 3DGS model renders high-quality synthetic images and depth maps to facilitate the establishment of 2D-3D correspondences.… ▽ More We leverage 3D Gaussian Splatting (3DGS) as a scene representation and propose a novel test-time camera pose refinement framework, GSLoc. This framework enhances the localization accuracy of state-of-the-art absolute pose regression and scene coordinate regression methods. The 3DGS model renders high-quality synthetic images and depth maps to facilitate the establishment of 2D-3D correspondences. GSLoc obviates the need for training feature extractors or descriptors by operating directly on RGB images, utilizing the 3D vision foundation model, MASt3R, for precise 2D matching. To improve the robustness of our model in challenging outdoor environments, we incorporate an exposure-adaptive module within the 3DGS framework. Consequently, GSLoc enables efficient pose refinement given a single RGB query and a coarse initial pose estimation. Our proposed approach surpasses leading NeRF-based optimization methods in both accuracy and runtime across indoor and outdoor visual localization benchmarks, achieving state-of-the-art accuracy on two indoor datasets. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Comments: The project page is available at https://gsloc.active.vision

AIR-HLoc: Adaptive Image Retrieval for Efficient Visual Localisation

Authors: Changkun Liu, Huajian Huang, Zhengyang Ma, Tristan Braud

Abstract: State-of-the-art (SOTA) hierarchical localisation pipelines (HLoc) rely on image retrieval (IR) techniques to establish 2D-3D correspondences by selecting the $k$ most similar images from a reference image database for a given query image. Although higher values of $k$ enhance localisation robustness, the computational cost for feature matching increases linearly with $k$. In this paper, we observ… ▽ More State-of-the-art (SOTA) hierarchical localisation pipelines (HLoc) rely on image retrieval (IR) techniques to establish 2D-3D correspondences by selecting the $k$ most similar images from a reference image database for a given query image. Although higher values of $k$ enhance localisation robustness, the computational cost for feature matching increases linearly with $k$. In this paper, we observe that queries that are the most similar to images in the database result in a higher proportion of feature matches and, thus, more accurate positioning. Thus, a small number of images is sufficient for queries very similar to images in the reference database. We then propose a novel approach, AIR-HLoc, which divides query images into different localisation difficulty levels based on their similarity to the reference image database. We consider an image with high similarity to the reference image as an easy query and an image with low similarity as a hard query. Easy queries show a limited improvement in accuracy when increasing $k$. Conversely, higher values of $k$ significantly improve accuracy for hard queries. Given the limited improvement in accuracy when increasing $k$ for easy queries and the significant improvement for hard queries, we adapt the value of $k$ to the query's difficulty level. Therefore, AIR-HLoc optimizes processing time by adaptively assigning different values of $k$ based on the similarity between the query and reference images without losing accuracy. Our extensive experiments on the Cambridge Landmarks, 7Scenes, and Aachen Day-Night-v1.1 datasets demonstrate our algorithm's efficacy, reducing 30\%, 26\%, and 11\% in computational overhead while maintaining SOTA accuracy compared to HLoc with fixed image retrieval. △ Less

Submitted 27 March, 2024; originally announced March 2024.

HR-APR: APR-agnostic Framework with Uncertainty Estimation and Hierarchical Refinement for Camera Relocalisation

Authors: Changkun Liu, Shuai Chen, Yukun Zhao, Huajian Huang, Victor Prisacariu, Tristan Braud

Abstract: Absolute Pose Regressors (APRs) directly estimate camera poses from monocular images, but their accuracy is unstable for different queries. Uncertainty-aware APRs provide uncertainty information on the estimated pose, alleviating the impact of these unreliable predictions. However, existing uncertainty modelling techniques are often coupled with a specific APR architecture, resulting in suboptimal… ▽ More Absolute Pose Regressors (APRs) directly estimate camera poses from monocular images, but their accuracy is unstable for different queries. Uncertainty-aware APRs provide uncertainty information on the estimated pose, alleviating the impact of these unreliable predictions. However, existing uncertainty modelling techniques are often coupled with a specific APR architecture, resulting in suboptimal performance compared to state-of-the-art (SOTA) APR methods. This work introduces a novel APR-agnostic framework, HR-APR, that formulates uncertainty estimation as cosine similarity estimation between the query and database features. It does not rely on or affect APR network architecture, which is flexible and computationally efficient. In addition, we take advantage of the uncertainty for pose refinement to enhance the performance of APR. The extensive experiments demonstrate the effectiveness of our framework, reducing 27.4\% and 15.2\% of computational overhead on the 7Scenes and Cambridge Landmarks datasets while maintaining the SOTA accuracy in single-image APRs. △ Less

Submitted 18 April, 2024; v1 submitted 22 February, 2024; originally announced February 2024.

Comments: Accepted in in 2024 IEEE International Conference on Robotics and Automation (ICRA). Code: https://github.com/lck666666/HR-APR

MobileARLoc: On-device Robust Absolute Localisation for Pervasive Markerless Mobile AR

Authors: Changkun Liu, Yukun Zhao, Tristan Braud

Abstract: Recent years have seen significant improvement in absolute camera pose estimation, paving the way for pervasive markerless Augmented Reality (AR). However, accurate absolute pose estimation techniques are computation- and storage-heavy, requiring computation offloading. As such, AR systems rely on visual-inertial odometry (VIO) to track the device's relative pose between requests to the server. Ho… ▽ More Recent years have seen significant improvement in absolute camera pose estimation, paving the way for pervasive markerless Augmented Reality (AR). However, accurate absolute pose estimation techniques are computation- and storage-heavy, requiring computation offloading. As such, AR systems rely on visual-inertial odometry (VIO) to track the device's relative pose between requests to the server. However, VIO suffers from drift, requiring frequent absolute repositioning. This paper introduces MobileARLoc, a new framework for on-device large-scale markerless mobile AR that combines an absolute pose regressor (APR) with a local VIO tracking system. Absolute pose regressors (APRs) provide fast on-device pose estimation at the cost of reduced accuracy. To address APR accuracy and reduce VIO drift, MobileARLoc creates a feedback loop where VIO pose estimations refine the APR predictions. The VIO system identifies reliable predictions of APR, which are then used to compensate for the VIO drift. We comprehensively evaluate MobileARLoc through dataset simulations. MobileARLoc halves the error compared to the underlying APR and achieve fast (80\,ms) on-device inference speed. △ Less

Submitted 4 February, 2024; v1 submitted 21 January, 2024; originally announced January 2024.

Comments: Accepted for publication at the 3rd edition of the Pervasive and Resource-Constrained AI (PerConAI) workshop (co-located with PerCom 2024). This article supersedes arXiv:2308.05394

360Loc: A Dataset and Benchmark for Omnidirectional Visual Localization with Cross-device Queries

Authors: Huajian Huang, Changkun Liu, Yipeng Zhu, Hui Cheng, Tristan Braud, Sai-Kit Yeung

Abstract: Portable 360$^\circ$ cameras are becoming a cheap and efficient tool to establish large visual databases. By capturing omnidirectional views of a scene, these cameras could expedite building environment models that are essential for visual localization. However, such an advantage is often overlooked due to the lack of valuable datasets. This paper introduces a new benchmark dataset, 360Loc, compos… ▽ More Portable 360$^\circ$ cameras are becoming a cheap and efficient tool to establish large visual databases. By capturing omnidirectional views of a scene, these cameras could expedite building environment models that are essential for visual localization. However, such an advantage is often overlooked due to the lack of valuable datasets. This paper introduces a new benchmark dataset, 360Loc, composed of 360$^\circ$ images with ground truth poses for visual localization. We present a practical implementation of 360$^\circ$ mapping combining 360$^\circ$ images with lidar data to generate the ground truth 6DoF poses. 360Loc is the first dataset and benchmark that explores the challenge of cross-device visual positioning, involving 360$^\circ$ reference frames, and query frames from pinhole, ultra-wide FoV fisheye, and 360$^\circ$ cameras. We propose a virtual camera approach to generate lower-FoV query frames from 360$^\circ$ images, which ensures a fair comparison of performance among different query types in visual localization tasks. We also extend this virtual camera approach to feature matching-based and pose regression-based methods to alleviate the performance loss caused by the cross-device domain gap, and evaluate its effectiveness against state-of-the-art baselines. We demonstrate that omnidirectional visual localization is more robust in challenging large-scale scenes with symmetries and repetitive structures. These results provide new insights into 360-camera mapping and omnidirectional visual localization with cross-device queries. △ Less

Submitted 31 May, 2024; v1 submitted 29 November, 2023; originally announced November 2023.

Comments: CVPR 2024. Project Page: https://huajianup.github.io/research/360Loc/

VR PreM+ : An Immersive Pre-learning Branching Visualization System for Museum Tours

Authors: Ze Gao, Xiang Li, Changkun Liu, Xian Wang, Anqi Wang, Liang Yang, Yuyang Wang, Pan Hui, Tristan Braud

Abstract: We present VR PreM+, an innovative VR system designed to enhance web exploration beyond traditional computer screens. Unlike static 2D displays, VR PreM+ leverages 3D environments to create an immersive pre-learning experience. Using keyword-based information retrieval allows users to manage and connect various content sources in a dynamic 3D space, improving communication and data comparison. We… ▽ More We present VR PreM+, an innovative VR system designed to enhance web exploration beyond traditional computer screens. Unlike static 2D displays, VR PreM+ leverages 3D environments to create an immersive pre-learning experience. Using keyword-based information retrieval allows users to manage and connect various content sources in a dynamic 3D space, improving communication and data comparison. We conducted preliminary and user studies that demonstrated efficient information retrieval, increased user engagement, and a greater sense of presence. These findings yielded three design guidelines for future VR information systems: display, interaction, and user-centric design. VR PreM+ bridges the gap between traditional web browsing and immersive VR, offering an interactive and comprehensive approach to information acquisition. It holds promise for research, education, and beyond. △ Less

Submitted 1 November, 2023; v1 submitted 20 October, 2023; originally announced October 2023.

Comments: Accepted for publication at The Eleventh International Symposium of Chinese CHI (Chinese CHI 2023), Bali

MSC Class: 14J60 (Primary) 14F05; 14J26 (Secondary) ACM Class: F.2.2; I.2.7

AI Nushu: An Exploration of Language Emergence in Sisterhood -Through the Lens of Computational Linguistics

Authors: Yuqian Sun, Yuying Tang, Ze Gao, Zhijun Pan, Chuyan Xu, Yurou Chen, Kejiang Qian, Zhigang Wang, Tristan Braud, Chang Hee Lee, Ali Asadipour

Abstract: This paper presents "AI Nushu," an emerging language system inspired by Nushu (women's scripts), the unique language created and used exclusively by ancient Chinese women who were thought to be illiterate under a patriarchal society. In this interactive installation, two artificial intelligence (AI) agents are trained in the Chinese dictionary and the Nushu corpus. By continually observing their e… ▽ More This paper presents "AI Nushu," an emerging language system inspired by Nushu (women's scripts), the unique language created and used exclusively by ancient Chinese women who were thought to be illiterate under a patriarchal society. In this interactive installation, two artificial intelligence (AI) agents are trained in the Chinese dictionary and the Nushu corpus. By continually observing their environment and communicating, these agents collaborate towards creating a standard writing system to encode Chinese. It offers an artistic interpretation of the creation of a non-western script from a computational linguistics perspective, integrating AI technology with Chinese cultural heritage and a feminist viewpoint. △ Less

Submitted 18 October, 2023; originally announced October 2023.

Comments: Accepted for publication at SIGGRAPH Asia 2023

MSC Class: 14J60 (Primary) 14F05; 14J26 (Secondary) ACM Class: F.2.2; I.2.7

KS-APR: Keyframe Selection for Robust Absolute Pose Regression

Authors: Changkun Liu, Yukun Zhao, Tristan Braud

Abstract: Markerless Mobile Augmented Reality (AR) aims to anchor digital content in the physical world without using specific 2D or 3D objects. Absolute Pose Regressors (APR) are end-to-end machine learning solutions that infer the device's pose from a single monocular image. Thanks to their low computation cost, they can be directly executed on the constrained hardware of mobile AR devices. However, APR m… ▽ More Markerless Mobile Augmented Reality (AR) aims to anchor digital content in the physical world without using specific 2D or 3D objects. Absolute Pose Regressors (APR) are end-to-end machine learning solutions that infer the device's pose from a single monocular image. Thanks to their low computation cost, they can be directly executed on the constrained hardware of mobile AR devices. However, APR methods tend to yield significant inaccuracies for input images that are too distant from the training set. This paper introduces KS-APR, a pipeline that assesses the reliability of an estimated pose with minimal overhead by combining the inference results of the APR and the prior images in the training set. Mobile AR systems tend to rely upon visual-inertial odometry to track the relative pose of the device during the experience. As such, KS-APR favours reliability over frequency, discarding unreliable poses. This pipeline can integrate most existing APR methods to improve accuracy by filtering unreliable images with their pose estimates. We implement the pipeline on three types of APR models on indoor and outdoor datasets. The median error on position and orientation is reduced for all models, and the proportion of large errors is minimized across datasets. Our method enables state-of-the-art APRs such as DFNetdm to outperform single-image and sequential APR methods. These results demonstrate the scalability and effectiveness of KS-APR for visual localization tasks that do not require one-shot decisions. △ Less

Submitted 28 April, 2024; v1 submitted 10 August, 2023; originally announced August 2023.

Robust Localization with Visual-Inertial Odometry Constraints for Markerless Mobile AR

Authors: Changkun Liu, Yukun Zhao, Tristan Braud

Abstract: Visual Inertial Odometry (VIO) is an essential component of modern Augmented Reality (AR) applications. However, VIO only tracks the relative pose of the device, leading to drift over time. Absolute pose estimation methods infer the device's absolute pose, but their accuracy depends on the input quality. This paper introduces VIO-APR, a new framework for markerless mobile AR that combines an absol… ▽ More Visual Inertial Odometry (VIO) is an essential component of modern Augmented Reality (AR) applications. However, VIO only tracks the relative pose of the device, leading to drift over time. Absolute pose estimation methods infer the device's absolute pose, but their accuracy depends on the input quality. This paper introduces VIO-APR, a new framework for markerless mobile AR that combines an absolute pose regressor (APR) with a local VIO tracking system. VIO-APR uses VIO to assess the reliability of the APR and the APR to identify and compensate for VIO drift. This feedback loop results in more accurate positioning and more stable AR experiences. To evaluate VIO-APR, we created a dataset that combines camera images with ARKit's VIO system output for six indoor and outdoor scenes of various scales. Over this dataset, VIO-APR improves the median accuracy of popular APR by up to 36\% in position and 29\% in orientation, increases the percentage of frames in the high ($0.25 m, 2^{\circ}$) accuracy level by up to 112\% and reduces the percentage of frames predicted below the low ($5 m, 10^\circ$) accuracy greatly. We implement VIO-APR into a mobile AR application using Unity to demonstrate its capabilities. VIO-APR results in noticeably more accurate localization and a more stable overall experience. △ Less

Submitted 15 September, 2023; v1 submitted 10 August, 2023; originally announced August 2023.

Tangible Web: An Interactive Immersion Virtual RealityCreativity System that Travels Across Reality

Authors: Simin Yang, Ze Gao, Reza Hadi Mogavi, Pan Hui, Tristan Braud

Abstract: With the advancement of virtual reality (VR) technology, virtual displays have become integral to how museums, galleries, and other tourist destinations present their collections to the public. However, the current lack of immersion in virtual reality displays limits the user's ability to experience and appreciate its aesthetics. This paper presents a case study of a creative approach taken by a t… ▽ More With the advancement of virtual reality (VR) technology, virtual displays have become integral to how museums, galleries, and other tourist destinations present their collections to the public. However, the current lack of immersion in virtual reality displays limits the user's ability to experience and appreciate its aesthetics. This paper presents a case study of a creative approach taken by a tourist attraction venue in developing a physical network system that allows visitors to enhance VR's aesthetic aspects based on environmental parameters gathered by external sensors. Our system was collaboratively developed through interviews and sessions with twelve stakeholder groups interested in art and exhibitions. This paper demonstrates how our technological advancements in interaction, immersion, and visual attractiveness surpass those of earlier virtual display generations. Through multimodal interaction, we aim to encourage innovation on the Web and create more visually appealing and engaging virtual displays. It is hoped that the greater online art community will gain fresh insight into how people interact with virtual worlds as a result of this work. △ Less

Submitted 5 April, 2023; originally announced April 2023.

Comments: Accepted In Proceedings of the ACM Web Conference 2023, April 30-May 4, 2023, Austin, TX, USA. ACM, New York, NY, USA

A Twitter Dataset for Pakistani Political Discourse

Authors: Ehsan-Ul Haq, Haris Bin Zia, Reza Hadi Mogavi, Gareth Tyson, Yang K. Lu, Tristan Braud, Pan Hui

Abstract: We share the largest dataset for the Pakistani Twittersphere consisting of over 49 million tweets, collected during one of the most politically active periods in the country. We collect the data after the deposition of the government by a No Confidence Vote in April 2022. This large-scale dataset can be used for several downstream tasks such as political bias, bots detection, trolling behavior, (d… ▽ More We share the largest dataset for the Pakistani Twittersphere consisting of over 49 million tweets, collected during one of the most politically active periods in the country. We collect the data after the deposition of the government by a No Confidence Vote in April 2022. This large-scale dataset can be used for several downstream tasks such as political bias, bots detection, trolling behavior, (dis)misinformation, and censorship related to Pakistani Twitter users. In addition, this dataset provides a large collection of tweets in Urdu and Roman Urdu that can be used for optimizing language processing tasks. △ Less

Submitted 16 January, 2023; originally announced January 2023.

Exploring Mental Health Communications among Instagram Coaches

Authors: Ehsan-Ul Haq, Lik-Hang Lee, Gareth Tyson, Reza Hadi Mogavi, Tristan Braud, Pan Hui

Abstract: There has been a significant expansion in the use of online social networks (OSNs) to support people experiencing mental health issues. This paper studies the role of Instagram influencers who specialize in coaching people with mental health issues. Using a dataset of 97k posts, we characterize such users' linguistic and behavioural features. We explore how these observations impact audience engag… ▽ More There has been a significant expansion in the use of online social networks (OSNs) to support people experiencing mental health issues. This paper studies the role of Instagram influencers who specialize in coaching people with mental health issues. Using a dataset of 97k posts, we characterize such users' linguistic and behavioural features. We explore how these observations impact audience engagement (as measured by likes). We show that the support provided by these accounts varies based on their self-declared professional identities. For instance, Instagram accounts that declare themselves as Authors offer less support than accounts that label themselves as Coach. We show that increasing information support in general communication positively affects user engagement. However, the effect of vocabulary on engagement is not consistent across the Instagram account types. Our findings shed light on this understudied topic and guide how mental health practitioners can improve outreach. △ Less

Submitted 11 November, 2022; originally announced November 2022.

Vetaverse: A Survey on the Intersection of Metaverse, Vehicles, and Transportation Systems

Authors: Pengyuan Zhou, Jinjing Zhu, Yiting Wang, Yunfan Lu, Zixiang Wei, Haolin Shi, Yuchen Ding, Yu Gao, Qinglong Huang, Yan Shi, Ahmad Alhilal, Lik-Hang Lee, Tristan Braud, Pan Hui, Lin Wang

Abstract: Since 2021, the term "Metaverse" has been the most popular one, garnering a lot of interest. Because of its contained environment and built-in computing and networking capabilities, a modern car makes an intriguing location to host its own little metaverse. Additionally, the travellers don't have much to do to pass the time while traveling, making them ideal customers for immersive services. Vetav… ▽ More Since 2021, the term "Metaverse" has been the most popular one, garnering a lot of interest. Because of its contained environment and built-in computing and networking capabilities, a modern car makes an intriguing location to host its own little metaverse. Additionally, the travellers don't have much to do to pass the time while traveling, making them ideal customers for immersive services. Vetaverse (Vehicular-Metaverse), which we define as the future continuum between vehicular industries and Metaverse, is envisioned as a blended immersive realm that scales up to cities and countries, as digital twins of the intelligent Transportation Systems, referred to as "TS-Metaverse", as well as customized XR services inside each Individual Vehicle, referred to as "IV-Metaverse". The two subcategories serve fundamentally different purposes, namely long-term interconnection, maintenance, monitoring, and management on scale for large transportation systems (TS), and personalized, private, and immersive infotainment services (IV). By outlining the framework of Vetaverse and examining important enabler technologies, we reveal this impending trend. Additionally, we examine unresolved issues and potential routes for future study while highlighting some intriguing Vetaverse services. △ Less

Submitted 16 March, 2023; v1 submitted 26 October, 2022; originally announced October 2022.

Comments: 27 pages, 19 figures

Beyond the Blue Sky of Multimodal Interaction: A Centennial Vision of Interplanetary Virtual Spaces in Turn-based Metaverse

Authors: Lik Hang Lee, Carlos Bermejo Fernandez, Ahmad Alhilal, Tristan Braud, Simo Hosio, Pan Hui, Esmée Henrieke Henrieke Anne de Haas

Abstract: Human habitation across multiple planets requires communication and social connection between planets. When the infrastructure of a deep space network becomes mature, immersive cyberspace, known as the Metaverse, can exchange diversified user data and host multitudinous virtual worlds. Nevertheless, such immersive cyberspace unavoidably encounters latency in minutes, and thus operates in a turn-ta… ▽ More Human habitation across multiple planets requires communication and social connection between planets. When the infrastructure of a deep space network becomes mature, immersive cyberspace, known as the Metaverse, can exchange diversified user data and host multitudinous virtual worlds. Nevertheless, such immersive cyberspace unavoidably encounters latency in minutes, and thus operates in a turn-taking manner. This Blue Sky paper illustrates a vision of an interplanetary Metaverse that connects Earthian and Martian users in a turn-based Metaverse. Accordingly, we briefly discuss several grand challenges to catalyze research initiatives for the `Digital Big Bang' on Mars. △ Less

Submitted 28 July, 2022; originally announced August 2022.

Comments: Accepted Paper in ACM ICMI 2022 (Blue Sky Track)

MSC Class: Nil ACM Class: A.1; K.0

Decentralized, not Dehumanized in the Metaverse: Bringing Utility to NFTs through Multimodal Interaction

Authors: Anqi Wang, Ze Gao, Lik-Hang Lee, Tristan Braud, Pan Hui

Abstract: User Interaction for NFTs (Non-fungible Tokens) is gaining increasing attention. Although NFTs have been traditionally single-use and monolithic, recent applications aim to connect multimodal interaction with human behaviour. This paper reviews the related technological approaches and business practices in NFT art. We highlight that multimodal interaction is a currently under-studied issue in main… ▽ More User Interaction for NFTs (Non-fungible Tokens) is gaining increasing attention. Although NFTs have been traditionally single-use and monolithic, recent applications aim to connect multimodal interaction with human behaviour. This paper reviews the related technological approaches and business practices in NFT art. We highlight that multimodal interaction is a currently under-studied issue in mainstream NFT art, and conjecture that multimodal interaction is a crucial enabler for decentralization in the NFT community. We propose a framework that combines a bottom-up approach with AI multimodal process. Through this framework, we put forward integrating human behaviour data into generative NFT units, as "multimodal interactive NFT". Our work displays the possibilities of NFTs in the art world, beyond the traditional 2D and 3D static content. △ Less

Submitted 8 June, 2022; originally announced June 2022.

Comments: 7 pages, 5 figures

What is the Metaverse? An Immersive Cyberspace and Open Challenges

Authors: Lik-Hang Lee, Pengyuan Zhou, Tristan Braud, Pan Hui

Abstract: The Metaverse refers to a virtual-physical blended space in which multiple users can concurrently interact with a unified computer-generated environment and other users, which can be regarded as the next significant milestone of the current cyberspace. This article primarily discusses the development and challenges of the Metaverse. We first briefly describe the development of cyberspace and the n… ▽ More The Metaverse refers to a virtual-physical blended space in which multiple users can concurrently interact with a unified computer-generated environment and other users, which can be regarded as the next significant milestone of the current cyberspace. This article primarily discusses the development and challenges of the Metaverse. We first briefly describe the development of cyberspace and the necessity of technology enablers. Accordingly, our bottom-up approach highlights three critical technology enablers for the Metaverse: networks, systems, and users. Also, we highlight a number of indispensable issues, under technological and ecosystem perspectives, that build and sustain the Metaverse. △ Less

Submitted 7 June, 2022; originally announced June 2022.

Comments: 7 pages, 2 figures

ACM Class: A.1; K.0

Re-shaping Post-COVID-19 Teaching and Learning: A Blueprint of Virtual-Physical Blended Classrooms in the Metaverse Era

Authors: Yuyang Wang, Lik-Hang Lee, Tristan Braud, Pan Hui

Abstract: During the COVID-19 pandemic, most countries have experienced some form of remote education through video conferencing software platforms. However, these software platforms fail to reduce immersion and replicate the classroom experience. The currently emerging Metaverse addresses many of such limitations by offering blended physical-digital environments. This paper aims to assess how the Metaverse… ▽ More During the COVID-19 pandemic, most countries have experienced some form of remote education through video conferencing software platforms. However, these software platforms fail to reduce immersion and replicate the classroom experience. The currently emerging Metaverse addresses many of such limitations by offering blended physical-digital environments. This paper aims to assess how the Metaverse can support and improve e-learning. We first survey the latest applications of blended environments in education and highlight the primary challenges and opportunities. Accordingly, we derive our proposal for a virtual-physical blended classroom configuration that brings students and teachers into a shared educational Metaverse. We focus on the system architecture of the Metaverse classroom to achieve real-time synchronization of a large number of participants and activities across physical (mixed reality classrooms) and virtual (remote VR platform) learning spaces. Our proposal attempts to transform the traditional physical classroom into virtual-physical cyberspace as a new social network of learners and educators connected at an unprecedented scale. △ Less

Submitted 28 April, 2022; v1 submitted 17 March, 2022; originally announced March 2022.

Comments: 6 pages, 3 figures, accepted in 2022 IEEE 42nd ICDCS Workshop on SocialMeta

ACM Class: K.3; I.3

Twitter Dataset for 2022 Russo-Ukrainian Crisis

Authors: Ehsan-Ul Haq, Gareth Tyson, Lik-Hang Lee, Tristan Braud, Pan Hui

Abstract: Online Social Networks (OSNs) play a significant role in information sharing during a crisis. The data collected during such a crisis can reflect the large scale public opinions and sentiment. In addition, OSN data can also be used to study different campaigns that are employed by various entities to engineer public opinions. Such information sharing campaigns can range from spreading factual info… ▽ More Online Social Networks (OSNs) play a significant role in information sharing during a crisis. The data collected during such a crisis can reflect the large scale public opinions and sentiment. In addition, OSN data can also be used to study different campaigns that are employed by various entities to engineer public opinions. Such information sharing campaigns can range from spreading factual information to propaganda and misinformation. We provide a Twitter dataset of the 2022 Russo-Ukrainian conflict. In the first release, we share over 1.6 million tweets shared during the 1st week of the crisis. △ Less

Submitted 6 March, 2022; originally announced March 2022.

From Digital Media to Empathic Reality: A Systematic Review of Empathy Research in Extended Reality Environments

Authors: Ville Paananen, Mohammad Sina Kiarostami, Lik-Hang Lee, Tristan Braud, Simo Hosio

Abstract: Recent advances in extended reality (XR) technologies have enabled new and increasingly realistic empathy tools and experiences. In XR, all interactions take place in different spatial contexts, all with different features, affordances, and constraints. We present a systematic literature survey of recent work on empathy in XR. As a result, we contribute a research roadmap with three future opportu… ▽ More Recent advances in extended reality (XR) technologies have enabled new and increasingly realistic empathy tools and experiences. In XR, all interactions take place in different spatial contexts, all with different features, affordances, and constraints. We present a systematic literature survey of recent work on empathy in XR. As a result, we contribute a research roadmap with three future opportunities in XR-enabled empathy research across both physical and virtual spaces. △ Less

Submitted 23 February, 2022; originally announced March 2022.

Nebula: Reliable Low-latency Video Transmission for Mobile Cloud Gaming

Authors: Ahmad Alhilal, Tristan Braud, Bo Han, Pan Hui

Abstract: Mobile cloud gaming enables high-end games on constrained devices by streaming the game content from powerful servers through mobile networks. Mobile networks suffer from highly variable bandwidth, latency, and losses that affect the gaming experience. This paper introduces Nebula, an end-to-end cloud gaming framework to minimize the impact of network conditions on the user experience. Nebula reli… ▽ More Mobile cloud gaming enables high-end games on constrained devices by streaming the game content from powerful servers through mobile networks. Mobile networks suffer from highly variable bandwidth, latency, and losses that affect the gaming experience. This paper introduces Nebula, an end-to-end cloud gaming framework to minimize the impact of network conditions on the user experience. Nebula relies on an end-to-end distortion model adapting the video source rate and the amount of frame-level redundancy based on the measured network conditions. As a result, it minimizes the motion-to-photon (MTP) latency while protecting the frames from losses. We fully implement Nebula and evaluate its performance against the state of the art techniques and latest research in real-time mobile cloud gaming transmission on a physical testbed over emulated and real wireless networks. Nebula consistently balances MTP latency (<140 ms) and visual quality (>31 dB) even in highly variable environments. A user experiment confirms that Nebula maximizes the user experience with high perceived video quality, playability, and low user load. △ Less

Submitted 16 January, 2022; originally announced January 2022.

Comments: 12, 13, accepted in conference: TheWebConf'22, uses subfig.dtx

ACM Class: C.2.4; H.5.1

DiOS -- An Extended Reality Operating System for the Metaverse

Authors: Tristan Braud, Lik-Hang Lee, Ahmad Alhilal, Carlos Bermejo Fernández, Pan Hui

Abstract: Driven by the recent improvements in device and networks capabilities, Extended Reality (XR) is becoming more pervasive; industry and academia alike envision ambitious projects such as the metaverse. However, XR is still limited by the current architecture of mobile systems. This paper makes the case for an XR-specific operating system (XROS). Such an XROS integrates hardware-support, computer vis… ▽ More Driven by the recent improvements in device and networks capabilities, Extended Reality (XR) is becoming more pervasive; industry and academia alike envision ambitious projects such as the metaverse. However, XR is still limited by the current architecture of mobile systems. This paper makes the case for an XR-specific operating system (XROS). Such an XROS integrates hardware-support, computer vision algorithms, and XR-specific networking as the primitives supporting XR technology. These primitives represent the physical-digital world as a single shared resource among applications. Such an XROS allows for the development of coherent and system-wide interaction and display methods, systematic privacy preservation on sensor data, and performance improvement while simplifying application development. △ Less

Submitted 10 January, 2022; originally announced January 2022.

Comments: 6 pages, 4 figures

EdgeXAR: A 6-DoF Camera Multi-target Interaction Framework for MAR with User-friendly Latency Compensation

Authors: Wenxiao Zhang, Sikun Lin, Farshid Hassani Bijarbooneh, Haofei Cheng, Tristan Braud, Pengyuan Zhou, Lik-hang Lee, Pan Hui

Abstract: The computational capabilities of recent mobile devices enable the processing of natural features for Augmented Reality (AR), but the scalability is still limited by the devices' computation power and available resources. In this paper, we propose EdgeXAR, a mobile AR framework that utilizes the advantages of edge computing through task offloading to support flexible camera-based AR interaction. W… ▽ More The computational capabilities of recent mobile devices enable the processing of natural features for Augmented Reality (AR), but the scalability is still limited by the devices' computation power and available resources. In this paper, we propose EdgeXAR, a mobile AR framework that utilizes the advantages of edge computing through task offloading to support flexible camera-based AR interaction. We propose a hybrid tracking system for mobile devices that provides lightweight tracking with 6 Degrees of Freedom and hides the offloading latency from users' perception. A practical, reliable and unreliable communication mechanism is used to achieve fast response and consistency of crucial information. We also propose a multi-object image retrieval pipeline that executes fast and accurate image recognition tasks on the cloud and edge servers. Extensive experiments are carried out to evaluate the performance of EdgeXAR by building mobile AR Apps upon it. Regarding the Quality of Experience (QoE), the mobile AR Apps powered by EdgeXAR framework run on average at the speed of 30 frames per second with precise tracking of only 1~2 pixel errors and accurate image recognition of at least 97% accuracy. As compared to Vuforia, one of the leading commercial AR frameworks, EdgeXAR transmits 87% less data while providing a stable 30 FPS performance and reducing the offloading latency by 50 to 70% depending on the transmission medium. Our work facilitates the large-scale deployment of AR as the next generation of ubiquitous interfaces. △ Less

Submitted 9 November, 2021; originally announced November 2021.

Comments: 24 pages, ACM EICS'22. arXiv admin note: substantial text overlap with arXiv:1805.03060

All One Needs to Know about Metaverse: A Complete Survey on Technological Singularity, Virtual Ecosystem, and Research Agenda

Authors: Lik-Hang Lee, Tristan Braud, Pengyuan Zhou, Lin Wang, Dianlei Xu, Zijun Lin, Abhishek Kumar, Carlos Bermejo, Pan Hui

Abstract: Since the popularisation of the Internet in the 1990s, the cyberspace has kept evolving. We have created various computer-mediated virtual environments including social networks, video conferencing, virtual 3D worlds (e.g., VR Chat), augmented reality applications (e.g., Pokemon Go), and Non-Fungible Token Games (e.g., Upland). Such virtual environments, albeit non-perpetual and unconnected, have… ▽ More Since the popularisation of the Internet in the 1990s, the cyberspace has kept evolving. We have created various computer-mediated virtual environments including social networks, video conferencing, virtual 3D worlds (e.g., VR Chat), augmented reality applications (e.g., Pokemon Go), and Non-Fungible Token Games (e.g., Upland). Such virtual environments, albeit non-perpetual and unconnected, have bought us various degrees of digital transformation. The term `metaverse' has been coined to further facilitate the digital transformation in every aspect of our physical lives. At the core of the metaverse stands the vision of an immersive Internet as a gigantic, unified, persistent, and shared realm. While the metaverse may seem futuristic, catalysed by emerging technologies such as Extended Reality, 5G, and Artificial Intelligence, the digital `big bang' of our cyberspace is not far away. This survey paper presents the first effort to offer a comprehensive framework that examines the latest metaverse development under the dimensions of state-of-the-art technologies and metaverse ecosystems, and illustrates the possibility of the digital `big bang'. First, technologies are the enablers that drive the transition from the current Internet to the metaverse. We thus examine eight enabling technologies rigorously - Extended Reality, User Interactivity (Human-Computer Interaction), Artificial Intelligence, Blockchain, Computer Vision, IoT and Robotics, Edge and Cloud computing, and Future Mobile Networks. In terms of applications, the metaverse ecosystem allows human users to live and play within a self-sustaining, persistent, and shared realm. Therefore, we discuss six user-centric factors -- Avatar, Content Creation, Virtual Economy, Social Acceptability, Security and Privacy, and Trust and Accountability. Finally, we propose a concrete research agenda for the development of the metaverse. △ Less

Submitted 3 November, 2021; v1 submitted 6 October, 2021; originally announced October 2021.

Comments: 66 pages

ACM Class: A.1; K.0

A Hierarchical State-Machine-Based Framework for Platoon Manoeuvre Descriptions

Authors: Corvin Deboeser, Jordan Ivanchev, Thomas Braud, Alois Knoll, David Eckhoff, Alberto Sangiovanni-Vincentelli

Abstract: This paper introduces the SEAD framework that simplifies the process of designing and describing autonomous vehicle platooning manoeuvres. Although a large body of research has been formulating platooning manoeuvres, it is still challenging to design, describe, read, and understand them. This difficulty largely arises from missing formalisation. To fill this gap, we analysed existing ways of descr… ▽ More This paper introduces the SEAD framework that simplifies the process of designing and describing autonomous vehicle platooning manoeuvres. Although a large body of research has been formulating platooning manoeuvres, it is still challenging to design, describe, read, and understand them. This difficulty largely arises from missing formalisation. To fill this gap, we analysed existing ways of describing manoeuvres, derived the causes of difficulty, and designed a framework that simplifies the manoeuvre design process. Alongside, a Manoeuvre Design Language was developed to structurally describe manoeuvres in a machine-readable format. Unlike state-of-the-art manoeuvre descriptions that require one state machine for every participating vehicle, the SEAD framework allows describing any manoeuvre from the single perspective of the platoon leader. %As a proof of concept, the proposed framework was implemented in the mixed traffic simulation environment BEHAVE for an autonomous highway scenario. Using this framework, we implemented several manoeuvres as they were described in literature. To demonstrate the applicability of the framework, an experiment was performed to evaluate the execution time performance of multiple alternatives of the Join-Middle manoeuvre. This proof-of-concept experiment revealed that the manoeuvre execution time can be reduced by 28 \% through parallelising various steps without considerable secondary effects. We hope that the SEAD framework will pave the way for further research in the area of new manoeuvre design and optimisation by largely simplifying and unifying platooning manoeuvre representation. △ Less

Submitted 12 April, 2021; originally announced April 2021.

AICP: Augmented Informative Cooperative Perception

Authors: Pengyuan Zhou, Pranvera Kortoci, Yui-Pan Yau, Tristan Braud, Xiujun Wang, Benjamin Finley, Lik-Hang Lee, Sasu Tarkoma, Jussi Kangasharju, Pan Hui

Abstract: Connected vehicles, whether equipped with advanced driver-assistance systems or fully autonomous, require human driver supervision and are currently constrained to visual information in their line-of-sight. A cooperative perception system among vehicles increases their situational awareness by extending their perception range. Existing solutions focus on improving perspective transformation and fa… ▽ More Connected vehicles, whether equipped with advanced driver-assistance systems or fully autonomous, require human driver supervision and are currently constrained to visual information in their line-of-sight. A cooperative perception system among vehicles increases their situational awareness by extending their perception range. Existing solutions focus on improving perspective transformation and fast information collection. However, such solutions fail to filter out large amounts of less relevant data and thus impose significant network and computation load. Moreover, presenting all this less relevant data can overwhelm the driver and thus actually hinder them. To address such issues, we present Augmented Informative Cooperative Perception (AICP), the first fast-filtering system which optimizes the informativeness of shared data at vehicles to improve the fused presentation. To this end, an informativeness maximization problem is presented for vehicles to select a subset of data to display to their drivers. Specifically, we propose (i) a dedicated system design with custom data structure and lightweight routing protocol for convenient data encapsulation, fast interpretation and transmission, and (ii) a comprehensive problem formulation and efficient fitness-based sorting algorithm to select the most valuable data to display at the application layer. We implement a proof-of-concept prototype of AICP with a bandwidth-hungry, latency-constrained real-life augmented reality application. The prototype adds only 12.6 milliseconds of latency to a current informativeness-unaware system. Next, we test the networking performance of AICP at scale and show that ACIP effectively filters out less relevant packets and decreases the channel busy time. △ Less

Submitted 25 February, 2022; v1 submitted 14 January, 2021; originally announced January 2021.

Comments: Accepted in IEEE Transactions on Intelligent Transportation Systems

DRLE: Decentralized Reinforcement Learning at the Edge for Traffic Light Control in the IoV

Authors: Pengyuan Zhou, Xianfu Chen, Zhi Liu, Tristan Braud, Pan Hui, Jussi Kangasharju

Abstract: The Internet of Vehicles (IoV) enables real-time data exchange among vehicles and roadside units and thus provides a promising solution to alleviate traffic jams in the urban area. Meanwhile, better traffic management via efficient traffic light control can benefit the IoV as well by enabling a better communication environment and decreasing the network load. As such, IoV and efficient traffic lig… ▽ More The Internet of Vehicles (IoV) enables real-time data exchange among vehicles and roadside units and thus provides a promising solution to alleviate traffic jams in the urban area. Meanwhile, better traffic management via efficient traffic light control can benefit the IoV as well by enabling a better communication environment and decreasing the network load. As such, IoV and efficient traffic light control can formulate a virtuous cycle. Edge computing, an emerging technology to provide low-latency computation capabilities at the edge of the network, can further improve the performance of this cycle. However, while the collected information is valuable, an efficient solution for better utilization and faster feedback has yet to be developed for edge-empowered IoV. To this end, we propose a Decentralized Reinforcement Learning at the Edge for traffic light control in the IoV (DRLE). DRLE exploits the ubiquity of the IoV to accelerate the collection of traffic data and its interpretation towards alleviating congestion and providing better traffic light control. DRLE operates within the coverage of the edge servers and uses aggregated data from neighboring edge servers to provide city-scale traffic light control. DRLE decomposes the highly complex problem of large area control. into a decentralized multi-agent problem. We prove its global optima with concrete mathematical reasoning. The proposed decentralized reinforcement learning algorithm running at each edge node adapts the traffic lights in real time. We conduct extensive evaluations and demonstrate the superiority of this approach over several state-of-the-art algorithms. △ Less

Submitted 5 January, 2021; v1 submitted 3 September, 2020; originally announced September 2020.

Comments: Accepted by IEEE Transactions on Intelligent Transportation Systems

Towards Augmented Reality-driven Human-City Interaction: Current Research on Mobile Headsets and Future Challenges

Authors: Lik Hang Lee, Tristan Braud, Simo Hosio, Pan Hui

Abstract: Interaction design for Augmented Reality (AR) is gaining increasing attention from both academia and industry. This survey discusses 260 articles (68.8% of articles published between 2015 - 2019) to review the field of human interaction in connected cities with emphasis on augmented reality-driven interaction. We provide an overview of Human-City Interaction and related technological approaches, f… ▽ More Interaction design for Augmented Reality (AR) is gaining increasing attention from both academia and industry. This survey discusses 260 articles (68.8% of articles published between 2015 - 2019) to review the field of human interaction in connected cities with emphasis on augmented reality-driven interaction. We provide an overview of Human-City Interaction and related technological approaches, followed by a review of the latest trends of information visualization, constrained interfaces, and embodied interaction for AR headsets. We highlight under-explored issues in interface design and input techniques that warrant further research, and conjecture that AR with complementary Conversational User Interfaces (CUIs) is a key enabler for ubiquitous interaction with immersive systems in smart cities. Our work helps researchers understand the current potential and future needs of AR in Human-City Interaction. △ Less

Submitted 25 May, 2021; v1 submitted 17 July, 2020; originally announced July 2020.

Comments: 39 pages, 12 figures, ACM Computing Survey (Accepted, May 2021)

MSC Class: 68-02 ACM Class: B.4; H.5

Marketplace for AI Models

Authors: Abhishek Kumar, Benjamin Finley, Tristan Braud, Sasu Tarkoma, Pan Hui

Abstract: Artificial intelligence shows promise for solving many practical societal problems in areas such as healthcare and transportation. However, the current mechanisms for AI model diffusion such as Github code repositories, academic project webpages, and commercial AI marketplaces have some limitations; for example, a lack of monetization methods, model traceability, and model auditabilty. In this wor… ▽ More Artificial intelligence shows promise for solving many practical societal problems in areas such as healthcare and transportation. However, the current mechanisms for AI model diffusion such as Github code repositories, academic project webpages, and commercial AI marketplaces have some limitations; for example, a lack of monetization methods, model traceability, and model auditabilty. In this work, we sketch guidelines for a new AI diffusion method based on a decentralized online marketplace. We consider the technical, economic, and regulatory aspects of such a marketplace including a discussion of solutions for problems in these areas. Finally, we include a comparative analysis of several current AI marketplaces that are already available or in development. We find that most of these marketplaces are centralized commercial marketplaces with relatively few models. △ Less

Submitted 3 March, 2020; originally announced March 2020.

Trustworthy AI in the Age of Pervasive Computing and Big Data

Authors: Abhishek Kumar, Tristan Braud, Sasu Tarkoma, Pan Hui

Abstract: The era of pervasive computing has resulted in countless devices that continuously monitor users and their environment, generating an abundance of user behavioural data. Such data may support improving the quality of service, but may also lead to adverse usages such as surveillance and advertisement. In parallel, Artificial Intelligence (AI) systems are being applied to sensitive fields such as he… ▽ More The era of pervasive computing has resulted in countless devices that continuously monitor users and their environment, generating an abundance of user behavioural data. Such data may support improving the quality of service, but may also lead to adverse usages such as surveillance and advertisement. In parallel, Artificial Intelligence (AI) systems are being applied to sensitive fields such as healthcare, justice, or human resources, raising multiple concerns on the trustworthiness of such systems. Trust in AI systems is thus intrinsically linked to ethics, including the ethics of algorithms, the ethics of data, or the ethics of practice. In this paper, we formalise the requirements of trustworthy AI systems through an ethics perspective. We specifically focus on the aspects that can be integrated into the design and development of AI systems. After discussing the state of research and the remaining challenges, we show how a concrete use-case in smart cities can benefit from these methods. △ Less

Submitted 30 January, 2020; originally announced February 2020.

Comments: To be published in Percrowd 2020 (PerCom Adjunct). Please cite as: Abhishek Kumar, Tristan Braud, Sasu Tarkoma, Pan Hui. Trustworthy AI in the Age of Pervasive Computing and Big Data. In Proceedings of the 3rd International Workshop on Context-awareness for Multi-device Pervasive and Mobile Computing (Percrowd), Austin USA, March 2020 (Percom 2020 Workshop)

Distributed Vehicular Computing at the Dawn of 5G: a Survey

Authors: Ahmad Alhilal, Benjamin Finley, Tristan Braud, Dongzhe Su, Pan Hui

Abstract: Recent advances in information technology have revolutionized the automotive industry, paving the way for next-generation smart vehicular mobility. Vehicles, roadside units, and other road users can collaborate to deliver novel services and applications. These services and applications require 1) massive volumes of heterogeneous and continuous data to perceive the environment, 2) reliable and low-… ▽ More Recent advances in information technology have revolutionized the automotive industry, paving the way for next-generation smart vehicular mobility. Vehicles, roadside units, and other road users can collaborate to deliver novel services and applications. These services and applications require 1) massive volumes of heterogeneous and continuous data to perceive the environment, 2) reliable and low-latency communication networks, 3) real-time data processing that provides decision support under application-specific constraints. Addressing such constraints introduces significant challenges for current communication and computing technologies. Relatedly, the fifth generation of cellular networks (5G) was developed to respond to communication challenges by providing for low-latency, high-reliability, and high bandwidth communications. As a major part of 5G, edge computing allows data offloading and computation at the edge of the network, ensuring low-latency and context-awareness, and 5G efficiency. In this work, we aim at providing a comprehensive overview of the state of research on vehicular computing in the emerging age of 5G and big data. △ Less

Submitted 25 April, 2022; v1 submitted 20 January, 2020; originally announced January 2020.

Comments: 24 pages, 11 figures, Journal

A Survey on Computational Politics

Authors: Ehsan ul Haq, Tristan Braud, Young D. Kwon, Pan Hui

Abstract: Computational Politics is the study of computational methods to analyze and moderate users' behaviors related to political activities such as election campaign persuasion, political affiliation, and opinion mining. With the rapid development and ease of access to the Internet, Information Communication Technologies (ICT) have given rise to massive numbers of users joining online communities and th… ▽ More Computational Politics is the study of computational methods to analyze and moderate users' behaviors related to political activities such as election campaign persuasion, political affiliation, and opinion mining. With the rapid development and ease of access to the Internet, Information Communication Technologies (ICT) have given rise to massive numbers of users joining online communities and the digitization of analogous data such as political debates. These communities and digitized data contain both explicit and latent information about users and their behaviors related to politics. For researchers, it is essential to utilize data from these sources to develop and design systems that not only provide solutions to computational politics but also help other businesses, such as marketers to increase users, participation and interactions. In this survey, we attempt to categorize main areas in computational politics and summarize the prominent studies in one place to better understand computational politics across different and multidimensional platforms. e.g., online social networks, online forums, and political debates. We then conclude this study by highlighting future research directions, opportunities, and challenges. △ Less

Submitted 2 April, 2020; v1 submitted 16 August, 2019; originally announced August 2019.

Future Architecture of the Interplanetary Internet

Authors: Ahmad Alhilal, Tristan Braud, Pan Hui

Abstract: Fifty years after the Apollo program, space exploration has recently been regaining popularity thanks to missions with high media coverage. Future space exploration and space station missions will require specific networks to interconnect Earth with other objects and planets in the solar system. The interconnections of these networks form the core of an Interplanetary Internet (IPN). More specific… ▽ More Fifty years after the Apollo program, space exploration has recently been regaining popularity thanks to missions with high media coverage. Future space exploration and space station missions will require specific networks to interconnect Earth with other objects and planets in the solar system. The interconnections of these networks form the core of an Interplanetary Internet (IPN). More specifically, we consider the IPN as the combination of physical infrastructure, network architecture, and technologies to provide communication and navigation services for missions and further applications. Compared to the current implementation of the Internet, nodes composing the core of the IPN are highly heterogeneous (base stations on planets, satellites etc.). Moreover, nodes are in constant motion over intersecting elliptical planes, which results in highly variable delays and even temporary unavailability of parts of the network. As such, an IPN has to overcome the challenges of conventional opportunistic networks, with much higher latency and jitter (from a couple of minutes to several days) and the additional constraint of long-term autonomous operations. In this paper, we highlight the challenges of IPN, demonstrate the elements to deploy within the areas of interest, and propose the technologies to handle deep space communication. We provide recommendations for an evolutionary IPN implementation, coherent with specific milestones of space exploration. △ Less

Submitted 2 October, 2018; originally announced October 2018.

Comments: 10 pages, 12 figures

Spatial Interference Detection for Mobile Visible Light Communication

Authors: Ali Ugur Guler, Tristan Braud, Pan Hui

Abstract: Taking advantage of the rolling shutter effect of CMOS cameras in smartphones is a common practice to increase the transfered data rate with visible light communication (VLC) without employing external equipment such as photodiodes. VLC can then be used as replacement of other marker based techniques for object identification for Augmented Reality and Ubiquitous computing applications. However, th… ▽ More Taking advantage of the rolling shutter effect of CMOS cameras in smartphones is a common practice to increase the transfered data rate with visible light communication (VLC) without employing external equipment such as photodiodes. VLC can then be used as replacement of other marker based techniques for object identification for Augmented Reality and Ubiquitous computing applications. However, the rolling shutter effect only allows to transmit data over a single dimension, which considerably limits the available bandwidth. In this article we propose a new method exploiting spacial interference detection to enable parallel transmission and design a protocol that enables easy identification of interferences between two signals. By introducing a second dimension, we are not only able to significantly increase the available bandwidth, but also identify and isolate light sources in close proximity. △ Less

Submitted 13 July, 2017; originally announced July 2017.