-
Collaborative Machine Learning Model Building with Families Using Co-ML
Authors:
Tiffany Tseng,
Jennifer King Chen,
Mona Abdelrahman,
Mary Beth Kery,
Fred Hohman,
Adriana Hilliard,
R. Benjamin Shapiro
Abstract:
Existing novice-friendly machine learning (ML) modeling tools center around a solo user experience, where a single user collects only their own data to build a model. However, solo modeling experiences limit valuable opportunities for encountering alternative ideas and approaches that can arise when learners work together; consequently, it often precludes encountering critical issues in ML around…
▽ More
Existing novice-friendly machine learning (ML) modeling tools center around a solo user experience, where a single user collects only their own data to build a model. However, solo modeling experiences limit valuable opportunities for encountering alternative ideas and approaches that can arise when learners work together; consequently, it often precludes encountering critical issues in ML around data representation and diversity that can surface when different perspectives are manifested in a group-constructed data set. To address this issue, we created Co-ML -- a tablet-based app for learners to collaboratively build ML image classifiers through an end-to-end, iterative model-building process. In this paper, we illustrate the feasibility and potential richness of collaborative modeling by presenting an in-depth case study of a family (two children 11 and 14-years-old working with their parents) using Co-ML in a facilitated introductory ML activity at home. We share the Co-ML system design and contribute a discussion of how using Co-ML in a collaborative activity enabled beginners to collectively engage with dataset design considerations underrepresented in prior work such as data diversity, class imbalance, and data quality. We discuss how a distributed collaborative process, in which individuals can take on different model-building responsibilities, provides a rich context for children and adults to learn ML dataset design.
△ Less
Submitted 14 June, 2023; v1 submitted 11 April, 2023;
originally announced April 2023.
-
Smartwatch-based ecological momentary assessments for occupant wellness and privacy in buildings
Authors:
Clayton Miller,
Renee Christensen,
Jin Kai Leong,
Mahmoud Abdelrahman,
Federico Tartarini,
Matias Quintana,
Andre Matthias Müller,
Mario Frei
Abstract:
This paper describes the adaptation of an open-source ecological momentary assessment smart-watch platform with three sets of micro-survey wellness-related questions focused on i) infectious disease (COVID-19) risk perception, ii) privacy and distraction in an office context, and iii) triggers of various movement-related behaviors in buildings. This platform was previously used to collect data for…
▽ More
This paper describes the adaptation of an open-source ecological momentary assessment smart-watch platform with three sets of micro-survey wellness-related questions focused on i) infectious disease (COVID-19) risk perception, ii) privacy and distraction in an office context, and iii) triggers of various movement-related behaviors in buildings. This platform was previously used to collect data for thermal comfort, and this work extends its use to other domains. Several research participants took part in a proof-of-concept experiment by wearing a smartwatch to collect their micro-survey question preferences and perception responses for two of the question sets. Participants were also asked to install an indoor localization app on their phone to detect where precisely in the building they completed the survey. The experiment identified occupant information such as the tendencies for the research participants to prefer privacy in certain spaces and the difference between infectious disease risk perception in naturally versus mechanically ventilated spaces.
△ Less
Submitted 11 August, 2022;
originally announced August 2022.
-
Targeting occupant feedback using digital twins: Adaptive spatial-temporal thermal preference sampling to optimize personal comfort models
Authors:
Mahmoud Abdelrahman,
Clayton Miller
Abstract:
Collecting intensive longitudinal thermal preference data from building occupants is emerging as an innovative means of characterizing the performance of buildings and the people who use them. These techniques have occupants giving subjective feedback using smartphones or smartwatches frequently over the course of days or weeks. The intention is that the data will be collected with high spatial an…
▽ More
Collecting intensive longitudinal thermal preference data from building occupants is emerging as an innovative means of characterizing the performance of buildings and the people who use them. These techniques have occupants giving subjective feedback using smartphones or smartwatches frequently over the course of days or weeks. The intention is that the data will be collected with high spatial and temporal diversity to best characterize a building and the occupant's preferences. But in reality, leaving the occupant to respond in an ad-hoc or fixed interval way creates unneeded survey fatigue and redundant data. This paper outlines a scenario-based (virtual experiment) method for optimizing data sampling using a smartwatch to achieve comparable accuracy in a personal thermal preference model with fewer data. This method uses BIM-extracted spatial data and Graph Neural Network-based (GNN) modeling to find regions of similar comfort preference to identify the best scenarios for triggering the occupant to give feedback. This method is compared to two baseline scenarios that use conventional zoning and a generic 4x4 square meter grid method from two field-based data sets. The results show that the proposed Build2Vec method has an 18-23\% higher overall sampling quality than the spaces-based and square-grid-based sampling methods. The Build2Vec method also performs similar to the baselines when removing redundant occupant feedback points but with better scalability potential.
△ Less
Submitted 10 April, 2022; v1 submitted 22 February, 2022;
originally announced February 2022.
-
Personal thermal comfort models using digital twins: Preference prediction with BIM-extracted spatial-temporal proximity data from Build2Vec
Authors:
Mahmoud Abdelrahman,
Adrian Chong,
Clayton Miller
Abstract:
Conventional thermal preference prediction in buildings has limitations due to the difficulty in capturing all environmental and personal factors. New model features can improve the ability of a machine learning model to classify a person's thermal preference. The spatial context of a building can provide information to models about the windows, walls, heating and cooling sources, air diffusers, a…
▽ More
Conventional thermal preference prediction in buildings has limitations due to the difficulty in capturing all environmental and personal factors. New model features can improve the ability of a machine learning model to classify a person's thermal preference. The spatial context of a building can provide information to models about the windows, walls, heating and cooling sources, air diffusers, and other factors that create micro-environments that influence thermal comfort. Due to spatial heterogeneity, it is impractical to position sensors at a high enough resolution to capture all conditions. This research aims to build upon an existing vector-based spatial model, called Build2Vec, for predicting spatial-temporal occupants' indoor environmental preferences. Build2Vec utilizes the spatial data from the Building Information Model (BIM) and indoor localization in a real-world setting. This framework uses longitudinal intensive thermal comfort subjective feedback from smart watch-based ecological momentary assessments (EMA). The aggregation of these data is combined into a graph network structure (i.e., objects and relations) and used as input for a classification model to predict occupant thermal preference. The results of a test implementation show 14-28% accuracy improvement over a set of baselines that use conventional thermal preference prediction input variables.
△ Less
Submitted 30 October, 2021;
originally announced November 2021.
-
Proactive Scheduling and Caching for Wireless VR Viewport Streaming
Authors:
Mostafa Abdelrahman,
Mohammed Elbamby,
Vilho Räisänen
Abstract:
Virtual Reality (VR) applications require high data rate for a high-quality immersive experience, in addition to low latency to avoid dizziness and motion sickness. One of the key wireless VR challenges is providing seamless connectivity and meeting the stringent latency and bandwidth requirements. This work proposes a proactive wireless VR system that utilizes information about the user's future…
▽ More
Virtual Reality (VR) applications require high data rate for a high-quality immersive experience, in addition to low latency to avoid dizziness and motion sickness. One of the key wireless VR challenges is providing seamless connectivity and meeting the stringent latency and bandwidth requirements. This work proposes a proactive wireless VR system that utilizes information about the user's future orientation for proactive scheduling and caching. This is achieved by leveraging deep neural networks to predict users' orientation trained on a real dataset. The 360° scene is then partitioned using an overlapping viewports scheme so that only portions of the scene covered by the users' perceptive field-of-view are streamed. Furthermore, to minimize the backhaul latency, popular viewports are cached at the edge cloud based on spatial popularity profiles. Through extensive simulations, we show that the proposed system provides significant latency and throughput performance improvement, especially in fluctuating channels and heavy load conditions. The proactive scheduling enabled by the combination of machine learning prediction and the proposed viewport scheme reduces the mean latency by more than 80% while achieving successful delivery rate close to 100%.
△ Less
Submitted 6 October, 2021;
originally announced October 2021.
-
Humans-as-a-sensor for buildings: Intensive longitudinal indoor comfort models
Authors:
Prageeth Jayathissa,
Matias Quintana,
Mahmoud Abdelrahman,
Clayton Miller
Abstract:
Evaluating and optimising human comfort within the built environment is challenging due to the large number of physiological, psychological and environmental variables that affect occupant comfort preference. Human perception could be helpful to capture these disparate phenomena and interpreting their impact; the challenge is collecting spatially and temporally diverse subjective feedback in a sca…
▽ More
Evaluating and optimising human comfort within the built environment is challenging due to the large number of physiological, psychological and environmental variables that affect occupant comfort preference. Human perception could be helpful to capture these disparate phenomena and interpreting their impact; the challenge is collecting spatially and temporally diverse subjective feedback in a scalable way. This paper presents a methodology to collect intensive longitudinal subjective feedback of comfort-based preference using micro ecological momentary assessments on a smartwatch platform. An experiment with 30 occupants over two weeks produced 4,378 field-based surveys for thermal, noise, and acoustic preference. The occupants and the spaces in which they left feedback were then clustered according to these preference tendencies. These groups were used to create different feature sets with combinations of environmental and physiological variables, for use in a multi-class classification task. These classification models were trained on a feature set that was developed from time-series attributes, environmental and near-body sensors, heart rate, and the historical preferences of both the individual and the comfort group assigned. The most accurate model had multi-class classification F1 micro scores of 64%, 80% and 86% for thermal, light, and noise preference, respectively. The discussion outlines how these models can enhance comfort preference prediction when supplementing data from installed sensors. The approach presented prompts reflection on how the building analysis community evaluates, controls, and designs indoor environments through balancing the measurement of variables with strategically asking for occupant preferences in an intensive longitudinal way.
△ Less
Submitted 25 August, 2020; v1 submitted 4 July, 2020;
originally announced July 2020.
-
Build2Vec: Building Representation in Vector Space
Authors:
Mahmoud Abdelrahman,
Adrian Chong,
Clayton Miller
Abstract:
In this paper, we represent a methodology of a graph embeddings algorithm that is used to transform labeled property graphs obtained from a Building Information Model (BIM). Industrial Foundation Classes (IFC) is a standard schema for BIM, which is utilized to convert the building data into a graph representation. We used node2Vec with biased random walks to extract semantic similarities between d…
▽ More
In this paper, we represent a methodology of a graph embeddings algorithm that is used to transform labeled property graphs obtained from a Building Information Model (BIM). Industrial Foundation Classes (IFC) is a standard schema for BIM, which is utilized to convert the building data into a graph representation. We used node2Vec with biased random walks to extract semantic similarities between different building components and represent them in a multi-dimensional vector space. A case study implementation is conducted on a net-zero-energy building located at the National University of Singapore (SDE4). This approach shows promising machine learning applications in capturing the semantic relations and similarities of different building objects, more specifically, spatial and spatio-temporal data.
△ Less
Submitted 1 July, 2020;
originally announced July 2020.
