Showing 1–50 of 86 results for author: Picard, R

Sub-millisecond Entanglement and iSWAP Gate between Molecular Qubits

Authors: Lewis R. B. Picard, Annie J. Park, Gabriel E. Patenotte, Samuel Gebretsadkan, David Wellnitz, Ana Maria Rey, Kang-Kuen Ni

Abstract: Quantum computation (QC) and simulation rely on long-lived qubits with controllable interactions. Early work in quantum computing made use of molecules because of their readily available intramolecular nuclear spin coupling and chemical shifts, along with mature nuclear magnetic resonance techniques. Subsequently, the pursuit of many physical platforms has flourished. Trapped polar molecules have… ▽ More Quantum computation (QC) and simulation rely on long-lived qubits with controllable interactions. Early work in quantum computing made use of molecules because of their readily available intramolecular nuclear spin coupling and chemical shifts, along with mature nuclear magnetic resonance techniques. Subsequently, the pursuit of many physical platforms has flourished. Trapped polar molecules have been proposed as a promising quantum computing platform, offering scalability and single-particle addressability while still leveraging inherent complexity and strong couplings of molecules. Recent progress in the single quantum state preparation and coherence of the hyperfine-rotational states of individually trapped molecules allows them to serve as promising qubits, with intermolecular dipolar interactions creating entanglement. However, universal two-qubit gates have not been demonstrated with molecules. Here, we harness intrinsic molecular resources to implement a two-qubit iSWAP gate using individually trapped $X^{1}Σ^{+}$ NaCs molecules. We characterize the innate dipolar interaction between rotational states and control its strength by tuning the polarization of the traps. By allowing the molecules to interact for 664 $μ$s at a distance of 1.9 $μ$m, we create a maximally entangled Bell state with a fidelity of 94(3)\%, following postselection to remove trials with empty traps. Using motion-rotation coupling, we measure residual excitation of the lowest few motional states along the axial trapping direction and find them to be the primary source of decoherence. Finally, we identify two non-interacting hyperfine states within the ground rotational level in which we encode a qubit. The interaction is toggled by transferring between interacting and non-interacting states to realize an iSWAP gate. We verify the gate performance by measuring its logical truth table. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Site-selective preparation and multi-state readout of molecules in optical tweezers

Authors: Lewis R. B. Picard, Gabriel E. Patenotte, Annie J. Park, Samuel F. Gebretsadkan, Kang-Kuen Ni

Abstract: Polar molecules are a quantum resource with rich internal structure that can be coherently controlled. The structure, however, also makes the state preparation and measurement (SPAM) of molecules challenging. We advance the SPAM of individual molecules assembled from constituent atoms trapped in optical tweezer arrays. Sites without NaCs molecules are eliminated using high-fidelity Cs atom detecti… ▽ More Polar molecules are a quantum resource with rich internal structure that can be coherently controlled. The structure, however, also makes the state preparation and measurement (SPAM) of molecules challenging. We advance the SPAM of individual molecules assembled from constituent atoms trapped in optical tweezer arrays. Sites without NaCs molecules are eliminated using high-fidelity Cs atom detection, increasing the peak molecule filling fraction of the array threefold. We site-selectively initialize the array in a rotational qubit subspace that is insensitive to differential AC Stark shifts from the optical tweezer. Lastly, we detect multiple rotational states per experimental cycle by imaging atoms after sequential state-selective dissociations. These demonstrations extend the SPAM capabilities of molecules for quantum information, simulation, and metrology. △ Less

Submitted 24 January, 2024; originally announced January 2024.

Adjoints of sums of m-accretive operators and applications to non-autonomous evolutionary equations

Authors: Rainer Picard, Sascha Trostorff, Marcus Waurick

Abstract: We provide certain compatibility conditions for m-accretive operators such that the adjoint of the sum is given by the closure of the sum of the respective adjoint. We revisit the proof of well-posedness of the abstract class of partial differential-algebraic equations known as evolutionary equations. We show that the general mechanism provided here can be applied to establish well-posedness for n… ▽ More We provide certain compatibility conditions for m-accretive operators such that the adjoint of the sum is given by the closure of the sum of the respective adjoint. We revisit the proof of well-posedness of the abstract class of partial differential-algebraic equations known as evolutionary equations. We show that the general mechanism provided here can be applied to establish well-posedness for non-autonomous evolutionary equations with $L_{\infty}$-coefficients thus not only generalising known results but opening up new directions other methods such as evolution families have a hard time to come by. △ Less

Submitted 22 December, 2023; originally announced December 2023.

Comments: 13 pages

MSC Class: 35P05 (Primary); 47D99; 35Q61; 35Q59; 35K05; 35L05 (Secondary)

Induced gravitational waves: the effect of first order tensor perturbations

Authors: Raphael Picard, Karim A. Malik

Abstract: Scalar induced gravitational waves contribute to the cosmological gravitational wave background. They can be related to the primordial density power spectrum produced towards the end of inflation and therefore are a convenient new tool to constrain models of inflation. These waves are sourced by terms quadratic in perturbations and hence appear at second order in cosmological perturbation theory.… ▽ More Scalar induced gravitational waves contribute to the cosmological gravitational wave background. They can be related to the primordial density power spectrum produced towards the end of inflation and therefore are a convenient new tool to constrain models of inflation. These waves are sourced by terms quadratic in perturbations and hence appear at second order in cosmological perturbation theory. While the focus of research so far was on purely scalar source terms we also study the effect of including first order tensor perturbations as an additional source. This gives rise to two additional source terms: a term quadratic in the tensor perturbations and a cross term involving mixed scalar and tensor perturbations. We present full analytical expressions for the spectral density of these new source terms and discuss their general behaviour. To illustrate the generation mechanism we study two toy models containing a peak on small scales. For these models we show that the scalar-tensor contribution becomes non-negligible compared to the scalar-scalar contribution on smaller scales. We also consider implications for future gravitational wave surveys. △ Less

Submitted 13 December, 2023; v1 submitted 24 November, 2023; originally announced November 2023.

Comments: 31 pages, 7 figures

Families of Annihilating Skew-Selfadjoint Operators and their Connection to Hilbert Complexes

Authors: Dirk Pauly, Rainer Picard

Abstract: In this short note we show that Hilbert complexes are strongly related to what we shall call annihilating sets of skew-selfadjoint operators. This provides for a new perspective on the classical topic of Hilbert complexes viewed as families of commuting normal operators. In this short note we show that Hilbert complexes are strongly related to what we shall call annihilating sets of skew-selfadjoint operators. This provides for a new perspective on the classical topic of Hilbert complexes viewed as families of commuting normal operators. △ Less

Submitted 24 July, 2024; v1 submitted 3 July, 2023; originally announced July 2023.

Comments: Key words and phrases: Hilbert complexes, de Rham complex, elasticity complex, biharmonic complex, factorisation of hyperbolic problems

Extended rotational coherence of polar molecules in an elliptically polarized trap

Authors: Annie J. Park, Lewis R. B. Picard, Gabriel E. Patenotte, Jessie T. Zhang, Till Rosenband, Kang-Kuen Ni

Abstract: We demonstrate long rotational coherence of individual polar molecules in the motional ground state of an optical trap. In the present, previously unexplored regime, the rotational eigenstates of molecules are dominantly quantized by trapping light rather than static fields, and the main source of decoherence is differential light shift. In an optical tweezer array of NaCs molecules, we achieve a… ▽ More We demonstrate long rotational coherence of individual polar molecules in the motional ground state of an optical trap. In the present, previously unexplored regime, the rotational eigenstates of molecules are dominantly quantized by trapping light rather than static fields, and the main source of decoherence is differential light shift. In an optical tweezer array of NaCs molecules, we achieve a three-orders-of-magnitude reduction in differential light shift by changing the trap's polarization from linear to a specific "magic" ellipticity. With spin-echo pulses, we measure a rotational coherence time of 62(3) ms (one pulse) and 250(40) ms (up to 72 pulses), surpassing the projected duration of resonant dipole-dipole entangling gates by orders of magnitude. △ Less

Submitted 12 June, 2023; originally announced June 2023.

Multipar-T: Multiparty-Transformer for Capturing Contingent Behaviors in Group Conversations

Authors: Dong Won Lee, Yubin Kim, Rosalind Picard, Cynthia Breazeal, Hae Won Park

Abstract: As we move closer to real-world AI systems, AI agents must be able to deal with multiparty (group) conversations. Recognizing and interpreting multiparty behaviors is challenging, as the system must recognize individual behavioral cues, deal with the complexity of multiple streams of data from multiple people, and recognize the subtle contingent social exchanges that take place amongst group membe… ▽ More As we move closer to real-world AI systems, AI agents must be able to deal with multiparty (group) conversations. Recognizing and interpreting multiparty behaviors is challenging, as the system must recognize individual behavioral cues, deal with the complexity of multiple streams of data from multiple people, and recognize the subtle contingent social exchanges that take place amongst group members. To tackle this challenge, we propose the Multiparty-Transformer (Multipar-T), a transformer model for multiparty behavior modeling. The core component of our proposed approach is the Crossperson Attention, which is specifically designed to detect contingent behavior between pairs of people. We verify the effectiveness of Multipar-T on a publicly available video-based group engagement detection benchmark, where it outperforms state-of-the-art approaches in average F-1 scores by 5.2% and individual class F-1 scores by up to 10.0%. Through qualitative analysis, we show that our Crossperson Attention module is able to discover contingent behavior. △ Less

Submitted 19 April, 2023; originally announced April 2023.

Comments: 7 pages, 4 figures, IJCAI

A Note on Some Non-Local Boundary Conditions and their Use in Connection with Beltrami Fields

Authors: Rainer Picard, Sascha Trostorff

Abstract: We consider two operators $A_{0},B_{0}$ between two Hilbert spaces satisfying $A_{0}\subseteq-B_{0}^{\ast}$ and $B_{0}\subseteq-A_{0}^{\ast}$ and inspect extensions $A^{\#}$ and $B^{\#}$ of $A_{0}$ and $B_{0}$, respectively, whose domain consists of those elements satisfying an abstract periodic boundary condition. The motivating example is the derivative on some interval, where the so-defined rea… ▽ More We consider two operators $A_{0},B_{0}$ between two Hilbert spaces satisfying $A_{0}\subseteq-B_{0}^{\ast}$ and $B_{0}\subseteq-A_{0}^{\ast}$ and inspect extensions $A^{\#}$ and $B^{\#}$ of $A_{0}$ and $B_{0}$, respectively, whose domain consists of those elements satisfying an abstract periodic boundary condition. The motivating example is the derivative on some interval, where the so-defined realisation gives the classical derivative with periodic boundary conditions. We derive necessary and sufficient conditions for the operator equality $A^{\#}=-\left(B^{\#}\right)^{\ast}$ and illustrate our findings by applications to the classical vector analytic operators $\mathrm{grad},\,\mathrm{div}$ and $\mathrm{curl}$. In particular, the realisation $\mathrm{curl}^{\#}$ naturally arises in the study of so-called Beltrami fields. △ Less

Submitted 5 July, 2023; v1 submitted 20 March, 2023; originally announced March 2023.

Comments: Revised Version

High resolution photoassociation spectroscopy of the excited $c^3Σ_{1}^+$ potential of $^{23}$Na$^{133}$Cs

Authors: Lewis R. B. Picard, Jessie T. Zhang, William B. Cairncross, Kenneth Wang, Gabriel E. Patenotte, Annie J. Park, Yichao Yu, Lee R. Liu, Jonathan D. Hood, Rosario González-Férez, Kang-Kuen Ni

Abstract: We report on photoassociation spectroscopy probing the $c^3Σ_{1}^+$ potential of the bi-alkali NaCs molecule, identifying eleven vibrational lines between $v' = 0$ and $v' = 25$ of the excited $c^3Σ_{1}^+$ potential, and resolving their rotational and hyperfine structure. The observed lines are assigned by fitting to an effective Hamiltonian model of the excited state structure with rotational and… ▽ More We report on photoassociation spectroscopy probing the $c^3Σ_{1}^+$ potential of the bi-alkali NaCs molecule, identifying eleven vibrational lines between $v' = 0$ and $v' = 25$ of the excited $c^3Σ_{1}^+$ potential, and resolving their rotational and hyperfine structure. The observed lines are assigned by fitting to an effective Hamiltonian model of the excited state structure with rotational and hyperfine constants as free parameters. We discuss unexpected broadening of select vibrational lines, and its possible link to strong spin-orbit coupling of the $c^3Σ_{1}^+$ potential with the nearby $b^3Π_1$ and $B^1Π_1$ manifolds. Finally we report use of the $v' = 22$ line as an intermediate state for two-photon transfer of weakly bound Feshbach molecules to the rovibrational ground state of the $X^1Σ^+$ manifold. △ Less

Submitted 17 February, 2023; originally announced February 2023.

Mixed Effects Random Forests for Personalised Predictions of Clinical Depression Severity

Authors: Robert A. Lewis, Asma Ghandeharioun, Szymon Fedor, Paola Pedrelli, Rosalind Picard, David Mischoulon

Abstract: This work demonstrates how mixed effects random forests enable accurate predictions of depression severity using multimodal physiological and digital activity data collected from an 8-week study involving 31 patients with major depressive disorder. We show that mixed effects random forests outperform standard random forests and personal average baselines when predicting clinical Hamilton Depressio… ▽ More This work demonstrates how mixed effects random forests enable accurate predictions of depression severity using multimodal physiological and digital activity data collected from an 8-week study involving 31 patients with major depressive disorder. We show that mixed effects random forests outperform standard random forests and personal average baselines when predicting clinical Hamilton Depression Rating Scale scores (HDRS_17). Compared to the latter baseline, accuracy is significantly improved for each patient by an average of 0.199-0.276 in terms of mean absolute error (p<0.05). This is noteworthy as these simple baselines frequently outperform machine learning methods in mental health prediction tasks. We suggest that this improved performance results from the ability of the mixed effects random forest to personalise model parameters to individuals in the dataset. However, we find that these improvements pertain exclusively to scenarios where labelled patient data are available to the model at training time. Investigating methods that improve accuracy when generalising to new patients is left as important future work. △ Less

Submitted 23 January, 2023; originally announced January 2023.

Comments: 9 pages

Computational Empathy Counteracts the Negative Effects of Anger on Creative Problem Solving

Authors: Matthew Groh, Craig Ferguson, Robert Lewis, Rosalind Picard

Abstract: How does empathy influence creative problem solving? We introduce a computational empathy intervention based on context-specific affective mimicry and perspective taking by a virtual agent appearing in the form of a well-dressed polar bear. In an online experiment with 1,006 participants randomly assigned to an emotion elicitation intervention (with a control elicitation condition and anger elicit… ▽ More How does empathy influence creative problem solving? We introduce a computational empathy intervention based on context-specific affective mimicry and perspective taking by a virtual agent appearing in the form of a well-dressed polar bear. In an online experiment with 1,006 participants randomly assigned to an emotion elicitation intervention (with a control elicitation condition and anger elicitation condition) and a computational empathy intervention (with a control virtual agent and an empathic virtual agent), we examine how anger and empathy influence participants' performance in solving a word game based on Wordle. We find participants who are assigned to the anger elicitation condition perform significantly worse on multiple performance metrics than participants assigned to the control condition. However, we find the empathic virtual agent counteracts the drop in performance induced by the anger condition such that participants assigned to both the empathic virtual agent and the anger condition perform no differently than participants in the control elicitation condition and significantly better than participants assigned to the control virtual agent and the anger elicitation condition. While empathy reduces the negative effects of anger, we do not find evidence that the empathic virtual agent influences performance of participants who are assigned to the control elicitation condition. By introducing a framework for computational empathy interventions and conducting a two-by-two factorial design randomized experiment, we provide rigorous, empirical evidence that computational empathy can counteract the negative effects of anger on creative problem solving. △ Less

Submitted 15 August, 2022; originally announced August 2022.

M-Accretive Realisations of Skew-Symmetric Operators

Authors: Rainer Picard, Sascha Trostorff

Abstract: We consider skew-symmetric operators $A_{0}$ on a Hilbert space $H$ and characterise all (nonlinear) m-accretive restrictions of $A:=-A_{0}^{\ast}$ in terms of the "deficiency spaces" $\ker(1\pm A)$. The results are illustrated by several examples and applied to a partial differential equation with an impedance type boundary condition. We consider skew-symmetric operators $A_{0}$ on a Hilbert space $H$ and characterise all (nonlinear) m-accretive restrictions of $A:=-A_{0}^{\ast}$ in terms of the "deficiency spaces" $\ker(1\pm A)$. The results are illustrated by several examples and applied to a partial differential equation with an impedance type boundary condition. △ Less

Submitted 12 July, 2022; v1 submitted 11 July, 2022; originally announced July 2022.

Comments: Remark 5.4 was added

Enriching the quantum toolbox of ultracold molecules with Rydberg atoms

Authors: Kenneth Wang, Conner P. Williams, Lewis R. B. Picard, Norman Y. Yao, Kang-Kuen Ni

Abstract: We describe a quantum information architecture consisting of a hybrid array of optically-trapped molecules and atoms. This design leverages the large transition dipole moments of Rydberg atoms to mediate fast, high-fidelity gates between qubits encoded in coherent molecular degrees of freedom. Error channels of detuning, decay, pulse area noise, and leakage to other molecular states are discussed.… ▽ More We describe a quantum information architecture consisting of a hybrid array of optically-trapped molecules and atoms. This design leverages the large transition dipole moments of Rydberg atoms to mediate fast, high-fidelity gates between qubits encoded in coherent molecular degrees of freedom. Error channels of detuning, decay, pulse area noise, and leakage to other molecular states are discussed. The molecule-Rydberg interaction can also be used to enable nondestructive molecule detection and rotational state readout. We consider a specific near-term implementation of this scheme using NaCs molecules and Cs Rydberg atoms, showing that it is possible to implement 300~ns gates with a potential fidelity of $> 99.9\%$. △ Less

Submitted 23 June, 2022; v1 submitted 11 April, 2022; originally announced April 2022.

Journal ref: PRX Quantum 3, 030339 (2022)

Human Detection of Political Speech Deepfakes across Transcripts, Audio, and Video

Authors: Matthew Groh, Aruna Sankaranarayanan, Nikhil Singh, Dong Young Kim, Andrew Lippman, Rosalind Picard

Abstract: Recent advances in technology for hyper-realistic visual and audio effects provoke the concern that deepfake videos of political speeches will soon be indistinguishable from authentic video recordings. The conventional wisdom in communication theory predicts people will fall for fake news more often when the same version of a story is presented as a video versus text. We conduct 5 pre-registered r… ▽ More Recent advances in technology for hyper-realistic visual and audio effects provoke the concern that deepfake videos of political speeches will soon be indistinguishable from authentic video recordings. The conventional wisdom in communication theory predicts people will fall for fake news more often when the same version of a story is presented as a video versus text. We conduct 5 pre-registered randomized experiments with 2,215 participants to evaluate how accurately humans distinguish real political speeches from fabrications across base rates of misinformation, audio sources, question framings, and media modalities. We find base rates of misinformation minimally influence discernment and deepfakes with audio produced by the state-of-the-art text-to-speech algorithms are harder to discern than the same deepfakes with voice actor audio. Moreover across all experiments, we find audio and visual information enables more accurate discernment than text alone: human discernment relies more on how something is said, the audio-visual cues, than what is said, the speech content. △ Less

Submitted 15 January, 2024; v1 submitted 25 February, 2022; originally announced February 2022.

An optical tweezer array of ground-state polar molecules

Authors: Jessie T. Zhang, Lewis R. B. Picard, William B. Cairncross, Kenneth Wang, Yichao Yu, Fang Fang, Kang-Kuen Ni

Abstract: Fully internal and motional state controlled and individually manipulable polar molecules are desirable for many quantum science applications leveraging the rich state space and intrinsic interactions of molecules. While prior efforts at assembling molecules from their constituent atoms individually trapped in optical tweezers achieved such a goal for exactly one molecule, here we extend the techn… ▽ More Fully internal and motional state controlled and individually manipulable polar molecules are desirable for many quantum science applications leveraging the rich state space and intrinsic interactions of molecules. While prior efforts at assembling molecules from their constituent atoms individually trapped in optical tweezers achieved such a goal for exactly one molecule, here we extend the technique to an array of five molecules, unlocking the ability to study molecular interactions. We detail the technical challenges and solutions inherent in scaling this system up. With parallel preparation and control of multiple molecules in hand, this platform now serves as a starting point to harness the vast resources and long-range dipolar interactions of molecules. △ Less

Submitted 2 December, 2021; originally announced December 2021.

Predicting Driver Self-Reported Stress by Analyzing the Road Scene

Authors: Cristina Bustos, Neska Elhaouij, Albert Sole-Ribalta, Javier Borge-Holthoefer, Agata Lapedriza, Rosalind Picard

Abstract: Several studies have shown the relevance of biosignals in driver stress recognition. In this work, we examine something important that has been less frequently explored: We develop methods to test if the visual driving scene can be used to estimate a drivers' subjective stress levels. For this purpose, we use the AffectiveROAD video recordings and their corresponding stress labels, a continuous hu… ▽ More Several studies have shown the relevance of biosignals in driver stress recognition. In this work, we examine something important that has been less frequently explored: We develop methods to test if the visual driving scene can be used to estimate a drivers' subjective stress levels. For this purpose, we use the AffectiveROAD video recordings and their corresponding stress labels, a continuous human-driver-provided stress metric. We use the common class discretization for stress, dividing its continuous values into three classes: low, medium, and high. We design and evaluate three computer vision modeling approaches to classify the driver's stress levels: (1) object presence features, where features are computed using automatic scene segmentation; (2) end-to-end image classification; and (3) end-to-end video classification. All three approaches show promising results, suggesting that it is possible to approximate the drivers' subjective stress from the information found in the visual scene. We observe that the video classification, which processes the temporal information integrated with the visual information, obtains the highest accuracy of $0.72$, compared to a random baseline accuracy of $0.33$ when tested on a set of nine drivers. △ Less

Submitted 27 September, 2021; originally announced September 2021.

A Structural Observation on port-Hamiltonian Systems

Authors: Rainer Picard, Sascha Trostorff, Bruce Watson, Marcus Waurick

Abstract: We study port-Hamiltonian systems on a familiy of intervals and characterise all boundary conditions leading to $m$-accretive realisations of the port-Hamiltonian operator and thus to generators of contractive semigroups. The proofs are based on a structural observation that the port-Hamiltonian operator can be transformed to the derivative on a familiy of reference intervals by suitable congruenc… ▽ More We study port-Hamiltonian systems on a familiy of intervals and characterise all boundary conditions leading to $m$-accretive realisations of the port-Hamiltonian operator and thus to generators of contractive semigroups. The proofs are based on a structural observation that the port-Hamiltonian operator can be transformed to the derivative on a familiy of reference intervals by suitable congruence relations allowing for studying the simpler case of a transport equation. Moreover, we provide well-posedness results for associated control problems without assuming any additional regularity of the operators involved. △ Less

Submitted 21 June, 2021; originally announced June 2021.

DISSECT: Disentangled Simultaneous Explanations via Concept Traversals

Authors: Asma Ghandeharioun, Been Kim, Chun-Liang Li, Brendan Jou, Brian Eoff, Rosalind W. Picard

Abstract: Explaining deep learning model inferences is a promising venue for scientific understanding, improving safety, uncovering hidden biases, evaluating fairness, and beyond, as argued by many scholars. One of the principal benefits of counterfactual explanations is allowing users to explore "what-if" scenarios through what does not and cannot exist in the data, a quality that many other forms of expla… ▽ More Explaining deep learning model inferences is a promising venue for scientific understanding, improving safety, uncovering hidden biases, evaluating fairness, and beyond, as argued by many scholars. One of the principal benefits of counterfactual explanations is allowing users to explore "what-if" scenarios through what does not and cannot exist in the data, a quality that many other forms of explanation such as heatmaps and influence functions are inherently incapable of doing. However, most previous work on generative explainability cannot disentangle important concepts effectively, produces unrealistic examples, or fails to retain relevant information. We propose a novel approach, DISSECT, that jointly trains a generator, a discriminator, and a concept disentangler to overcome such challenges using little supervision. DISSECT generates Concept Traversals (CTs), defined as a sequence of generated examples with increasing degrees of concepts that influence a classifier's decision. By training a generative model from a classifier's signal, DISSECT offers a way to discover a classifier's inherent "notion" of distinct concepts automatically rather than rely on user-predefined concepts. We show that DISSECT produces CTs that (1) disentangle several concepts, (2) are influential to a classifier's decision and are coupled to its reasoning due to joint training (3), are realistic, (4) preserve relevant information, and (5) are stable across similar inputs. We validate DISSECT on several challenging synthetic and realistic datasets where previous methods fall short of satisfying desirable criteria for interpretability and show that it performs consistently well and better than existing methods. Finally, we present experiments showing applications of DISSECT for detecting potential biases of a classifier and identifying spurious artifacts that impact predictions. △ Less

Submitted 15 March, 2022; v1 submitted 31 May, 2021; originally announced May 2021.

Comments: Accepted for publication at ICLR 2022

Deepfake Detection by Human Crowds, Machines, and Machine-informed Crowds

Authors: Matthew Groh, Ziv Epstein, Chaz Firestone, Rosalind Picard

Abstract: The recent emergence of machine-manipulated media raises an important societal question: how can we know if a video that we watch is real or fake? In two online studies with 15,016 participants, we present authentic videos and deepfakes and ask participants to identify which is which. We compare the performance of ordinary human observers against the leading computer vision deepfake detection mode… ▽ More The recent emergence of machine-manipulated media raises an important societal question: how can we know if a video that we watch is real or fake? In two online studies with 15,016 participants, we present authentic videos and deepfakes and ask participants to identify which is which. We compare the performance of ordinary human observers against the leading computer vision deepfake detection model and find them similarly accurate while making different kinds of mistakes. Together, participants with access to the model's prediction are more accurate than either alone, but inaccurate model predictions often decrease participants' accuracy. To probe the relative strengths and weaknesses of humans and machines as detectors of deepfakes, we examine human and machine performance across video-level features, and we evaluate the impact of pre-registered randomized interventions on deepfake detection. We find that manipulations designed to disrupt visual processing of faces hinder human participants' performance while mostly not affecting the model's performance, suggesting a role for specialized cognitive capacities in explaining human deepfake detection performance. △ Less

Submitted 27 October, 2021; v1 submitted 13 May, 2021; originally announced May 2021.

Journal ref: Proceedings of the National Academy of Sciences Jan 2022, 119 (1) e2110013119

Personalized Federated Deep Learning for Pain Estimation From Face Images

Authors: Ognjen Rudovic, Nicolas Tobis, Sebastian Kaltwang, Björn Schuller, Daniel Rueckert, Jeffrey F. Cohn, Rosalind W. Picard

Abstract: Standard machine learning approaches require centralizing the users' data in one computer or a shared database, which raises data privacy and confidentiality concerns. Therefore, limiting central access is important, especially in healthcare settings, where data regulations are strict. A potential approach to tackling this is Federated Learning (FL), which enables multiple parties to collaborative… ▽ More Standard machine learning approaches require centralizing the users' data in one computer or a shared database, which raises data privacy and confidentiality concerns. Therefore, limiting central access is important, especially in healthcare settings, where data regulations are strict. A potential approach to tackling this is Federated Learning (FL), which enables multiple parties to collaboratively learn a shared prediction model by using parameters of locally trained models while keeping raw training data locally. In the context of AI-assisted pain-monitoring, we wish to enable confidentiality-preserving and unobtrusive pain estimation for long-term pain-monitoring and reduce the burden on the nursing staff who perform frequent routine check-ups. To this end, we propose a novel Personalized Federated Deep Learning (PFDL) approach for pain estimation from face images. PFDL performs collaborative training of a deep model, implemented using a lightweight CNN architecture, across different clients (i.e., subjects) without sharing their face images. Instead of sharing all parameters of the model, as in standard FL, PFDL retains the last layer locally (used to personalize the pain estimates). This (i) adds another layer of data confidentiality, making it difficult for an adversary to infer pain levels of the target subject, while (ii) personalizing the pain estimation to each subject through local parameter tuning. We show using a publicly available dataset of face videos of pain (UNBC-McMaster Shoulder Pain Database), that PFDL performs comparably or better than the standard centralized and FL algorithms, while further enhancing data privacy. This, has the potential to improve traditional pain monitoring by making it more secure, computationally efficient, and scalable to a large number of individuals (e.g., for in-home pain monitoring), providing timely and unobtrusive pain measurement. △ Less

Submitted 12 January, 2021; originally announced January 2021.

Comments: 12 pages, 6 figures

Assembly of a rovibrational ground state molecule in an optical tweezer

Authors: William B. Cairncross, Jessie T. Zhang, Lewis R. B. Picard, Yichao Yu, Kenneth Wang, Kang-Kuen Ni

Abstract: We demonstrate the coherent creation of a single NaCs molecule in its rotational, vibrational, and electronic (rovibronic) ground state in an optical tweezer. Starting with a weakly bound Feshbach molecule, we locate a two-photon transition via the $|{c^3Σ,v'=26}\rangle$ excited state and drive coherent Rabi oscillations between the Feshbach state and a single hyperfine level of the NaCs rovibroni… ▽ More We demonstrate the coherent creation of a single NaCs molecule in its rotational, vibrational, and electronic (rovibronic) ground state in an optical tweezer. Starting with a weakly bound Feshbach molecule, we locate a two-photon transition via the $|{c^3Σ,v'=26}\rangle$ excited state and drive coherent Rabi oscillations between the Feshbach state and a single hyperfine level of the NaCs rovibronic ground state $|{X^1Σ,v''=0,N''=0}\rangle$ with a binding energy of $D_0 = h \times 147038.30(2)$ GHz. We measure a lifetime of $3.4\pm1.6$ s for the rovibronic ground-state molecule, which possesses a large molecule-frame dipole moment of 4.6 Debye and occupies predominantly the motional ground state. These long-lived, fully quantum-state-controlled individual dipolar molecules provide a key resource for molecule-based quantum simulation and information processing. △ Less

Submitted 8 January, 2021; originally announced January 2021.

Journal ref: Phys. Rev. Lett. 126, 123402 (2021)

Coherent optical creation of a single molecule

Authors: Yichao Yu, Kenneth Wang, Jonathan D. Hood, Lewis R. B. Picard, Jessie T. Zhang, William B. Cairncross, Jeremy M. Hutson, Rosario Gonzalez-Ferez, Till Rosenband, Kang-Kuen Ni

Abstract: We report coherent association of atoms into a single weakly bound NaCs molecule in an optical tweezer through an optical Raman transition. The Raman technique uses a deeply bound electronic excited intermediate state to achieve a large transition dipole moment while reducing photon scattering. Starting from two atoms in their relative motional ground state, we achieve an optical transfer efficien… ▽ More We report coherent association of atoms into a single weakly bound NaCs molecule in an optical tweezer through an optical Raman transition. The Raman technique uses a deeply bound electronic excited intermediate state to achieve a large transition dipole moment while reducing photon scattering. Starting from two atoms in their relative motional ground state, we achieve an optical transfer efficiency of 69%. The molecules have a binding energy of 770.2MHz at 8.83(2)G. This technique does not rely on Feshbach resonances or narrow excited-state lines and may allow a wide range of molecular species to be assembled atom-by-atom. △ Less

Submitted 16 December, 2020; originally announced December 2020.

Journal ref: Phys. Rev. X 11, 031061 (2021)

Human-centric Dialog Training via Offline Reinforcement Learning

Authors: Natasha Jaques, Judy Hanwen Shen, Asma Ghandeharioun, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Shane Gu, Rosalind Picard

Abstract: How can we train a dialog model to produce better conversations by learning from human feedback, without the risk of humans teaching it harmful chat behaviors? We start by hosting models online, and gather human feedback from real-time, open-ended conversations, which we then use to train and improve the models using offline reinforcement learning (RL). We identify implicit conversational cues inc… ▽ More How can we train a dialog model to produce better conversations by learning from human feedback, without the risk of humans teaching it harmful chat behaviors? We start by hosting models online, and gather human feedback from real-time, open-ended conversations, which we then use to train and improve the models using offline reinforcement learning (RL). We identify implicit conversational cues including language similarity, elicitation of laughter, sentiment, and more, which indicate positive human feedback, and embed these in multiple reward functions. A well-known challenge is that learning an RL policy in an offline setting usually fails due to the lack of ability to explore and the tendency to make over-optimistic estimates of future reward. These problems become even harder when using RL for language models, which can easily have a 20,000 action vocabulary and many possible reward functions. We solve the challenge by developing a novel class of offline RL algorithms. These algorithms use KL-control to penalize divergence from a pre-trained prior language model, and use a new strategy to make the algorithm pessimistic, instead of optimistic, in the face of uncertainty. We test the resulting dialog model with ratings from 80 users in an open-domain setting and find it achieves significant improvements over existing deep offline RL approaches. The novel offline RL method is viable for improving any existing generative dialog model using a static dataset of human feedback. △ Less

Submitted 12 October, 2020; originally announced October 2020.

Comments: To appear in EMNLP 2020 (long paper)

Dynamic First Order Wave Systems with Drift Term on Riemannian Manifolds

Authors: Rainer Picard, Sascha Trostorff

Abstract: An abstract first order differential equation of hyperbolic type with drift term on a Riemannian manifold is considered. For proving its well-posedness, transmutator and commutator relations are needed, which are studied in a general functional analytic setting. An abstract first order differential equation of hyperbolic type with drift term on a Riemannian manifold is considered. For proving its well-posedness, transmutator and commutator relations are needed, which are studied in a general functional analytic setting. △ Less

Submitted 30 September, 2020; originally announced September 2020.

A Robotic Positive Psychology Coach to Improve College Students' Wellbeing

Authors: Sooyeon Jeong, Sharifa Alghowinem, Laura Aymerich-Franch, Kika Arias, Agata Lapedriza, Rosalind Picard, Hae Won Park, Cynthia Breazeal

Abstract: A significant number of college students suffer from mental health issues that impact their physical, social, and occupational outcomes. Various scalable technologies have been proposed in order to mitigate the negative impact of mental health disorders. However, the evaluation for these technologies, if done at all, often reports mixed results on improving users' mental health. We need to better… ▽ More A significant number of college students suffer from mental health issues that impact their physical, social, and occupational outcomes. Various scalable technologies have been proposed in order to mitigate the negative impact of mental health disorders. However, the evaluation for these technologies, if done at all, often reports mixed results on improving users' mental health. We need to better understand the factors that align a user's attributes and needs with technology-based interventions for positive outcomes. In psychotherapy theory, therapeutic alliance and rapport between a therapist and a client is regarded as the basis for therapeutic success. In prior works, social robots have shown the potential to build rapport and a working alliance with users in various settings. In this work, we explore the use of a social robot coach to deliver positive psychology interventions to college students living in on-campus dormitories. We recruited 35 college students to participate in our study and deployed a social robot coach in their room. The robot delivered daily positive psychology sessions among other useful skills like delivering the weather forecast, scheduling reminders, etc. We found a statistically significant improvement in participants' psychological wellbeing, mood, and readiness to change behavior for improved wellbeing after they completed the study. Furthermore, students' personality traits were found to have a significant association with intervention efficacy. Analysis of the post-study interview revealed students' appreciation of the robot's companionship and their concerns for privacy. △ Less

Submitted 8 September, 2020; originally announced September 2020.

Comments: 8 pages, 5 figures, RO-MAN 2020, Best paper award

Coherent Dynamics in Quantum Emitters under Dichromatic Excitation

Authors: Z. X. Koong, E. Scerri, M. Rambach, M. Cygorek, M. Brotons-Gisbert, R. Picard, Y. Ma, S. I. Park, J. D. Song, E. M. Gauger, B. D. Gerardot

Abstract: We characterize the coherent dynamics of a two-level quantum emitter driven by a pair of symmetrically-detuned phase-locked pulses. The promise of dichromatic excitation is to spectrally isolate the excitation laser from the quantum emission, enabling background-free photon extraction from the emitter. Paradoxically, we find that excitation is not possible without spectral overlap between the exci… ▽ More We characterize the coherent dynamics of a two-level quantum emitter driven by a pair of symmetrically-detuned phase-locked pulses. The promise of dichromatic excitation is to spectrally isolate the excitation laser from the quantum emission, enabling background-free photon extraction from the emitter. Paradoxically, we find that excitation is not possible without spectral overlap between the exciting pulse and the quantum emitter transition for ideal two-level systems due to cancellation of the accumulated pulse area. However, any additional interactions that interfere with cancellation of the accumulated pulse area may lead to a finite stationary population inversion. Our spectroscopic results of a solid-state two-level system show that while coupling to lattice vibrations helps to improve the inversion efficiency up to 50\% under symmetric driving, coherent population control and a larger amount of inversion are possible using asymmetric dichromatic excitation, which we achieve by adjusting the ratio of the intensities between the red and blue-detuned pulses. Our measured results, supported by simulations using a real-time path-integral method, offer a new perspective towards realising efficient, background-free photon generation and extraction. △ Less

Submitted 4 September, 2020; originally announced September 2020.

Comments: 16 pages

Journal ref: Phys. Rev. Lett. 126, 047403 (2021)

Dyadic Speech-based Affect Recognition using DAMI-P2C Parent-child Multimodal Interaction Dataset

Authors: Huili Chen, Yue Zhang, Felix Weninger, Rosalind Picard, Cynthia Breazeal, Hae Won Park

Abstract: Automatic speech-based affect recognition of individuals in dyadic conversation is a challenging task, in part because of its heavy reliance on manual pre-processing. Traditional approaches frequently require hand-crafted speech features and segmentation of speaker turns. In this work, we design end-to-end deep learning methods to recognize each person's affective expression in an audio stream wit… ▽ More Automatic speech-based affect recognition of individuals in dyadic conversation is a challenging task, in part because of its heavy reliance on manual pre-processing. Traditional approaches frequently require hand-crafted speech features and segmentation of speaker turns. In this work, we design end-to-end deep learning methods to recognize each person's affective expression in an audio stream with two speakers, automatically discovering features and time regions relevant to the target speaker's affect. We integrate a local attention mechanism into the end-to-end architecture and compare the performance of three attention implementations -- one mean pooling and two weighted pooling methods. Our results show that the proposed weighted-pooling attention solutions are able to learn to focus on the regions containing target speaker's affective information and successfully extract the individual's valence and arousal intensity. Here we introduce and use a "dyadic affect in multimodal interaction - parent to child" (DAMI-P2C) dataset collected in a study of 34 families, where a parent and a child (3-7 years old) engage in reading storybooks together. In contrast to existing public datasets for affect recognition, each instance for both speakers in the DAMI-P2C dataset is annotated for the perceived affect by three labelers. To encourage more research on the challenging task of multi-speaker affect sensing, we make the annotated DAMI-P2C dataset publicly available, including acoustic features of the dyads' raw audios, affect annotations, and a diverse set of developmental, social, and demographic profiles of each dyad. △ Less

Submitted 20 August, 2020; originally announced August 2020.

Comments: Accepted by the 2020 International Conference on Multimodal Interaction (ICMI'20)

ACM Class: I.2.0

openXDATA: A Tool for Multi-Target Data Generation and Missing Label Completion

Authors: Felix Weninger, Yue Zhang, Rosalind W. Picard

Abstract: A common problem in machine learning is to deal with datasets with disjoint label spaces and missing labels. In this work, we introduce the openXDATA tool that completes the missing labels in partially labelled or unlabelled datasets in order to generate multi-target data with labels in the joint label space of the datasets. To this end, we designed and implemented the cross-data label completion… ▽ More A common problem in machine learning is to deal with datasets with disjoint label spaces and missing labels. In this work, we introduce the openXDATA tool that completes the missing labels in partially labelled or unlabelled datasets in order to generate multi-target data with labels in the joint label space of the datasets. To this end, we designed and implemented the cross-data label completion (CDLC) algorithm that uses a multi-task shared-hidden-layer DNN to iteratively complete the sparse label matrix of the instances from the different datasets. We apply the new tool to estimate labels across four emotion datasets: one labeled with discrete emotion categories (e.g., happy, sad, angry), one labeled with continuous values along arousal and valence dimensions, one with both kinds of labels, and one unlabeled. Testing with drop-out of true labels, we show the ability to estimate both categories and continuous labels for all of the datasets, at rates that approached the ground truth values. openXDATA is available under the GNU General Public License from https://github.com/fweninger/openXDATA. △ Less

Submitted 27 July, 2020; originally announced July 2020.

Forming a single molecule by magnetoassociation in an optical tweezer

Authors: Jessie T. Zhang, Yichao Yu, William B. Cairncross, Kenneth Wang, Lewis R. B. Picard, Jonathan D. Hood, Yen-Wei Lin, Jeremy M. Hutson, Kang-Kuen Ni

Abstract: We demonstrate the formation of a single NaCs molecule in an optical tweezer by magnetoassociation through an s-wave Feshbach resonance at 864.11(5)G. Starting from single atoms cooled to their motional ground states, we achieve conversion efficiencies of 47(1)%, and measure a molecular lifetime of 4.7(7)ms. By construction, the single molecules are predominantly (77(5)%) in the center-of-mass mot… ▽ More We demonstrate the formation of a single NaCs molecule in an optical tweezer by magnetoassociation through an s-wave Feshbach resonance at 864.11(5)G. Starting from single atoms cooled to their motional ground states, we achieve conversion efficiencies of 47(1)%, and measure a molecular lifetime of 4.7(7)ms. By construction, the single molecules are predominantly (77(5)%) in the center-of-mass motional ground state of the tweezer. Furthermore, we produce a single p-wave molecule near 807G by first preparing one of the atoms with one quantum of motional excitation. Our creation of a single weakly bound molecule in a designated internal state in the motional ground state of an optical tweezer is a crucial step towards coherent control of single molecules in optical tweezer arrays. △ Less

Submitted 27 June, 2020; v1 submitted 17 March, 2020; originally announced March 2020.

Journal ref: Phys. Rev. Lett. 124, 253401 (2020)

Resonance fluorescence from waveguide-coupled strain-localized two-dimensional quantum emitters

Authors: Carlos Errando-Herranz, Eva Schöll, Raphaël Picard, Micaela Laini, Samuel Gyger, Ali W. Elshaari, Art Branny, Ulrika Wennberg, Sebastien Barbat, Thibaut Renaud, Mauro Brotons-Gisbert, Cristian Bonato, Brian D. Gerardot, Val Zwiller, Klaus D. Jöns

Abstract: Efficient on-chip integration of single-photon emitters imposes a major bottleneck for applications of photonic integrated circuits in quantum technologies. Resonantly excited solid-state emitters are emerging as near-optimal quantum light sources, if not for the lack of scalability of current devices. Current integration approaches rely on cost-inefficient individual emitter placement in photonic… ▽ More Efficient on-chip integration of single-photon emitters imposes a major bottleneck for applications of photonic integrated circuits in quantum technologies. Resonantly excited solid-state emitters are emerging as near-optimal quantum light sources, if not for the lack of scalability of current devices. Current integration approaches rely on cost-inefficient individual emitter placement in photonic integrated circuits, rendering applications impossible. A promising scalable platform is based on two-dimensional (2D) semiconductors. However, resonant excitation and single-photon emission of waveguide-coupled 2D emitters have proven to be elusive. Here, we show a scalable approach using a silicon nitride photonic waveguide to simultaneously strain-localize single-photon emitters from a tungsten diselenide (WSe2) monolayer and to couple them into a waveguide mode. We demonstrate the guiding of single photons in the photonic circuit by measuring second-order autocorrelation of g$^{(2)}(0)=0.150\pm0.093$ and perform on-chip resonant excitation yielding a g$^{(2)}(0)=0.377\pm0.081$. Our results are an important step to enable coherent control of quantum states and multiplexing of high-quality single photons in a scalable photonic quantum circuit. △ Less

Submitted 15 May, 2020; v1 submitted 18 February, 2020; originally announced February 2020.

Characterizing Sources of Uncertainty to Proxy Calibration and Disambiguate Annotator and Data Bias

Authors: Asma Ghandeharioun, Brian Eoff, Brendan Jou, Rosalind W. Picard

Abstract: Supporting model interpretability for complex phenomena where annotators can legitimately disagree, such as emotion recognition, is a challenging machine learning task. In this work, we show that explicitly quantifying the uncertainty in such settings has interpretability benefits. We use a simple modification of a classical network inference using Monte Carlo dropout to give measures of epistemic… ▽ More Supporting model interpretability for complex phenomena where annotators can legitimately disagree, such as emotion recognition, is a challenging machine learning task. In this work, we show that explicitly quantifying the uncertainty in such settings has interpretability benefits. We use a simple modification of a classical network inference using Monte Carlo dropout to give measures of epistemic and aleatoric uncertainty. We identify a significant correlation between aleatoric uncertainty and human annotator disagreement ($r\approx.3$). Additionally, we demonstrate how difficult and subjective training samples can be identified using aleatoric uncertainty and how epistemic uncertainty can reveal data bias that could result in unfair predictions. We identify the total uncertainty as a suitable surrogate for model calibration, i.e. the degree we can trust model's predicted confidence. In addition to explainability benefits, we observe modest performance boosts from incorporating model uncertainty. △ Less

Submitted 5 October, 2019; v1 submitted 19 September, 2019; originally announced September 2019.

Comments: Accepted for presentation at 2019 ICCV Workshop on Interpreting and Explaining Visual Artificial Intelligence Models

A Hilbert space approach to fractional differential equations

Authors: Kai Diethelm, Konrad Kitzing, Rainer Picard, Stefan Siegmund, Sascha Trostorff, Marcus Waurick

Abstract: We study fractional differential equations of Riemann-Liouville and Caputo type in Hilbert spaces. Using exponentially weighted spaces of functions defined on $\mathbb{R}$, we define fractional operators by means of a functional calculus using the Fourier transform. Main tools are extrapolation- and interpolation spaces. Main results are the existence and uniqueness of solutions and the causality… ▽ More We study fractional differential equations of Riemann-Liouville and Caputo type in Hilbert spaces. Using exponentially weighted spaces of functions defined on $\mathbb{R}$, we define fractional operators by means of a functional calculus using the Fourier transform. Main tools are extrapolation- and interpolation spaces. Main results are the existence and uniqueness of solutions and the causality of solution operators for non-linear fractional differential equations. △ Less

Submitted 29 January, 2020; v1 submitted 17 September, 2019; originally announced September 2019.

MSC Class: 26A33

Hierarchical Reinforcement Learning for Open-Domain Dialog

Authors: Abdelrhman Saleh, Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen, Rosalind Picard

Abstract: Open-domain dialog generation is a challenging problem; maximum likelihood training can lead to repetitive outputs, models have difficulty tracking long-term conversational goals, and training on standard movie or online datasets may lead to the generation of inappropriate, biased, or offensive text. Reinforcement Learning (RL) is a powerful framework that could potentially address these issues, f… ▽ More Open-domain dialog generation is a challenging problem; maximum likelihood training can lead to repetitive outputs, models have difficulty tracking long-term conversational goals, and training on standard movie or online datasets may lead to the generation of inappropriate, biased, or offensive text. Reinforcement Learning (RL) is a powerful framework that could potentially address these issues, for example by allowing a dialog model to optimize for reducing toxicity and repetitiveness. However, previous approaches which apply RL to open-domain dialog generation do so at the word level, making it difficult for the model to learn proper credit assignment for long-term conversational rewards. In this paper, we propose a novel approach to hierarchical reinforcement learning, VHRL, which uses policy gradients to tune the utterance-level embedding of a variational sequence model. This hierarchical approach provides greater flexibility for learning long-term, conversational rewards. We use self-play and RL to optimize for a set of human-centered conversation metrics, and show that our approach provides significant improvements -- in terms of both human evaluation and automatic metrics -- over state-of-the-art dialog models, including Transformers. △ Less

Submitted 31 December, 2019; v1 submitted 16 September, 2019; originally announced September 2019.

Pain Detection with fNIRS-Measured Brain Signals: A Personalized Machine Learning Approach Using the Wavelet Transform and Bayesian Hierarchical Modeling with Dirichlet Process Priors

Authors: Daniel Lopez-Martinez, Ke Peng, Arielle Lee, David Borsook, Rosalind Picard

Abstract: Currently self-report pain ratings are the gold standard in clinical pain assessment. However, the development of objective automatic measures of pain could substantially aid pain diagnosis and therapy. Recent neuroimaging studies have shown the potential of functional near-infrared spectroscopy (fNIRS) for pain detection. This is a brain-imaging technique that provides non-invasive, long-term mea… ▽ More Currently self-report pain ratings are the gold standard in clinical pain assessment. However, the development of objective automatic measures of pain could substantially aid pain diagnosis and therapy. Recent neuroimaging studies have shown the potential of functional near-infrared spectroscopy (fNIRS) for pain detection. This is a brain-imaging technique that provides non-invasive, long-term measurements of cortical hemoglobin concentration changes. In this study, we focused on fNIRS signals acquired exclusively from the prefrontal cortex, which can be accessed unobtrusively, and derived an algorithm for the detection of the presence of pain using Bayesian hierarchical modelling with wavelet features. This approach allows personalization of the inference process by accounting for inter-participant variability in pain responses. Our work highlights the importance of adopting a personalized approach and supports the use of fNIRS for pain assessment. △ Less

Submitted 30 July, 2019; originally announced July 2019.

Tweet Moodifier: Towards giving emotional awareness to Twitter users

Authors: Belen Saldias, Rosalind W. Picard

Abstract: Emotional contagion in online social networks has been of great interest over the past years. Previous studies have focused mainly on finding evidence of affect contagion in homophilic atmospheres. However, these studies have overlooked users' awareness of the sentiments they share and consume online. In this paper, we present an experiment with Twitter users that aims to help them better understa… ▽ More Emotional contagion in online social networks has been of great interest over the past years. Previous studies have focused mainly on finding evidence of affect contagion in homophilic atmospheres. However, these studies have overlooked users' awareness of the sentiments they share and consume online. In this paper, we present an experiment with Twitter users that aims to help them better understand which emotions they experience on this social network. We introduce Tweet Moodifier (T-Moodifier), a Google Chrome extension that enables Twitter users to filter and make explicit (through colored visual marks) the emotional content in their News Feed. We compare behavioral changes between 55 participants and 5089 of their public "friends." The comparison period spans from two weeks before installing T-Moodifier to one week thereafter. The results suggest that the use of T-Moodifier might help Twitter users increase their emotional awareness: T-Moodifier users who had access to emotional statistics about their posts produced a significantly higher percentage of neutral content. This behavioral change suggests that people could behave differently while using real-time mechanisms that increase their affect reflection. Also, post-experience, those who completed both pre- and post-surveys could assert more confidently the main emotions they shared and perceived on Twitter. This shows T-Moodifier's potential to effectively make users reflect on their News Feed. △ Less

Submitted 5 December, 2019; v1 submitted 26 July, 2019; originally announced July 2019.

Comments: Accepted at 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)

Journal ref: 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)

Detection of Real-world Driving-induced Affective State Using Physiological Signals and Multi-view Multi-task Machine Learning

Authors: Daniel Lopez-Martinez, Neska El-Haouij, Rosalind Picard

Abstract: Affective states have a critical role in driving performance and safety. They can degrade driver situation awareness and negatively impact cognitive processes, severely diminishing road safety. Therefore, detecting and assessing drivers' affective states is crucial in order to help improve the driving experience, and increase safety, comfort and well-being. Recent advances in affective computing h… ▽ More Affective states have a critical role in driving performance and safety. They can degrade driver situation awareness and negatively impact cognitive processes, severely diminishing road safety. Therefore, detecting and assessing drivers' affective states is crucial in order to help improve the driving experience, and increase safety, comfort and well-being. Recent advances in affective computing have enabled the detection of such states. This may lead to empathic automotive user interfaces that account for the driver's emotional state and influence the driver in order to improve safety. In this work, we propose a multiview multi-task machine learning method for the detection of driver's affective states using physiological signals. The proposed approach is able to account for inter-drive variability in physiological responses while enabling interpretability of the learned models, a factor that is especially important in systems deployed in the real world. We evaluate the models on three different datasets containing real-world driving experiences. Our results indicate that accounting for drive-specific differences significantly improves model performance. △ Less

Submitted 19 July, 2019; originally announced July 2019.

Comments: Affective Computing and Intelligent Interaction Conference 2019

Engineering Music to Slow Breathing and Invite Relaxed Physiology

Authors: Grace Leslie, Asma Ghandeharioun, Diane Y. Zhou, Rosalind W. Picard

Abstract: We engineered an interactive music system that influences a user's breathing rate to induce a relaxation response. This system generates ambient music containing periodic shifts in loudness that are determined by the user's own breathing patterns. We evaluated the efficacy of this music intervention for participants who were engaged in an attention-demanding task, and thus explicitly not focusing… ▽ More We engineered an interactive music system that influences a user's breathing rate to induce a relaxation response. This system generates ambient music containing periodic shifts in loudness that are determined by the user's own breathing patterns. We evaluated the efficacy of this music intervention for participants who were engaged in an attention-demanding task, and thus explicitly not focusing on their breathing or on listening to the music. We measured breathing patterns in addition to multiple peripheral and cortical indicators of physiological arousal while users experienced three different interaction designs: (1) a "Fixed Tempo" amplitude modulation rate at six beats per minute; (2) a "Personalized Tempo" modulation rate fixed at 75\% of each individual's breathing rate baseline, and (3) a "Personalized Envelope" design in which the amplitude modulation matches each individual's breathing pattern in real-time. Our results revealed that each interactive music design slowed down breathing rates, with the "Personalized Tempo" design having the largest effect, one that was more significant than the non-personalized design. The physiological arousal indicators (electrodermal activity, heart rate, and slow cortical potentials measured in EEG) showed concomitant reductions, suggesting that slowing users' breathing rates shifted them towards a more calmed state. These results suggest that interactive music incorporating biometric data may have greater effects on physiology than traditional recorded music. △ Less

Submitted 20 July, 2019; originally announced July 2019.

Comments: Accepted at 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)

Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialog

Authors: Natasha Jaques, Asma Ghandeharioun, Judy Hanwen Shen, Craig Ferguson, Agata Lapedriza, Noah Jones, Shixiang Gu, Rosalind Picard

Abstract: Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. These are critical shortcomings for applying RL to real-world problems where collecting data is expensive, and models must be tested offline before being deployed to interact with the environment -- e.g. systems that learn from human int… ▽ More Most deep reinforcement learning (RL) systems are not able to learn effectively from off-policy data, especially if they cannot explore online in the environment. These are critical shortcomings for applying RL to real-world problems where collecting data is expensive, and models must be tested offline before being deployed to interact with the environment -- e.g. systems that learn from human interaction. Thus, we develop a novel class of off-policy batch RL algorithms, which are able to effectively learn offline, without exploring, from a fixed batch of human interaction data. We leverage models pre-trained on data as a strong prior, and use KL-control to penalize divergence from this prior during RL training. We also use dropout-based uncertainty estimates to lower bound the target Q-values as a more efficient alternative to Double Q-Learning. The algorithms are tested on the problem of open-domain dialog generation -- a challenging reinforcement learning problem with a 20,000-dimensional action space. Using our Way Off-Policy algorithm, we can extract multiple different reward functions post-hoc from collected human interaction data, and learn effectively from all of these. We test the real-world generalization of these systems by deploying them live to converse with humans in an open-domain setting, and demonstrate that our algorithm achieves significant improvements over prior methods in off-policy batch RL. △ Less

Submitted 8 July, 2019; v1 submitted 30 June, 2019; originally announced July 2019.

Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems

Authors: Asma Ghandeharioun, Judy Hanwen Shen, Natasha Jaques, Craig Ferguson, Noah Jones, Agata Lapedriza, Rosalind Picard

Abstract: Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; we then introduce a novel, model-agnostic, and dataset-agnostic method to approx… ▽ More Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of static conversation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; we then introduce a novel, model-agnostic, and dataset-agnostic method to approximate it. In particular, we propose a self-play scenario where the dialog system talks to itself and we calculate a combination of proxies such as sentiment and semantic coherence on the conversation trajectory. We show that this metric is capable of capturing the human-rated quality of a dialog model better than any automated metric known to-date, achieving a significant Pearson correlation (r>.7, p<.05). To investigate the strengths of this novel metric and interactive evaluation in comparison to state-of-the-art metrics and human evaluation of static conversations, we perform extended experiments with a set of models, including several that make novel improvements to recent hierarchical dialog generation architectures through sentiment and semantic knowledge distillation on the utterance level. Finally, we open-source the interactive evaluation platform we built and the dataset we collected to allow researchers to efficiently deploy and evaluate dialog models. △ Less

Submitted 3 November, 2019; v1 submitted 21 June, 2019; originally announced June 2019.

Comments: 33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada

Multi-modal Active Learning From Human Data: A Deep Reinforcement Learning Approach

Authors: Ognjen Rudovic, Meiru Zhang, Bjorn Schuller, Rosalind W. Picard

Abstract: Human behavior expression and experience are inherently multi-modal, and characterized by vast individual and contextual heterogeneity. To achieve meaningful human-computer and human-robot interactions, multi-modal models of the users states (e.g., engagement) are therefore needed. Most of the existing works that try to build classifiers for the users states assume that the data to train the model… ▽ More Human behavior expression and experience are inherently multi-modal, and characterized by vast individual and contextual heterogeneity. To achieve meaningful human-computer and human-robot interactions, multi-modal models of the users states (e.g., engagement) are therefore needed. Most of the existing works that try to build classifiers for the users states assume that the data to train the models are fully labeled. Nevertheless, data labeling is costly and tedious, and also prone to subjective interpretations by the human coders. This is even more pronounced when the data are multi-modal (e.g., some users are more expressive with their facial expressions, some with their voice). Thus, building models that can accurately estimate the users states during an interaction is challenging. To tackle this, we propose a novel multi-modal active learning (AL) approach that uses the notion of deep reinforcement learning (RL) to find an optimal policy for active selection of the users data, needed to train the target (modality-specific) models. We investigate different strategies for multi-modal data fusion, and show that the proposed model-level fusion coupled with RL outperforms the feature-level and modality-specific models, and the naive AL strategies such as random sampling, and the standard heuristics such as uncertainty sampling. We show the benefits of this approach on the task of engagement estimation from real-world child-robot interactions during an autism therapy. Importantly, we show that the proposed multi-modal AL approach can be used to efficiently personalize the engagement classifiers to the target user using a small amount of actively selected users data. △ Less

Submitted 7 June, 2019; originally announced June 2019.

Deep Reinforcement Learning for Optimal Critical Care Pain Management with Morphine using Dueling Double-Deep Q Networks

Authors: Daniel Lopez-Martinez, Patrick Eschenfeldt, Sassan Ostvar, Myles Ingram, Chin Hur, Rosalind Picard

Abstract: Opioids are the preferred medications for the treatment of pain in the intensive care unit. While undertreatment leads to unrelieved pain and poor clinical outcomes, excessive use of opioids puts patients at risk of experiencing multiple adverse effects. In this work, we present a sequential decision making framework for opioid dosing based on deep reinforcement learning. It provides real-time cli… ▽ More Opioids are the preferred medications for the treatment of pain in the intensive care unit. While undertreatment leads to unrelieved pain and poor clinical outcomes, excessive use of opioids puts patients at risk of experiencing multiple adverse effects. In this work, we present a sequential decision making framework for opioid dosing based on deep reinforcement learning. It provides real-time clinically interpretable dosing recommendations, personalized according to each patient's evolving pain and physiological condition. We focus on morphine, one of the most commonly prescribed opioids. To train and evaluate the model, we used retrospective data from the publicly available MIMIC-3 database. Our results demonstrate that reinforcement learning may be used to aid decision making in the intensive care setting by providing personalized pain management interventions. △ Less

Submitted 24 April, 2019; originally announced April 2019.

Comments: 2019 41st Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC)

Meta-Weighted Gaussian Process Experts for Personalized Forecasting of AD Cognitive Changes

Authors: Ognjen Rudovic, Yuria Utsumi, Ricardo Guerrero, Kelly Peterson, Daniel Rueckert, Rosalind W. Picard

Abstract: We introduce a novel personalized Gaussian Process Experts (pGPE) model for predicting per-subject ADAS-Cog13 cognitive scores -- a significant predictor of Alzheimer's Disease (AD) in the cognitive domain -- over the future 6, 12, 18, and 24 months. We start by training a population-level model using multi-modal data from previously seen subjects using a base Gaussian Process (GP) regression. The… ▽ More We introduce a novel personalized Gaussian Process Experts (pGPE) model for predicting per-subject ADAS-Cog13 cognitive scores -- a significant predictor of Alzheimer's Disease (AD) in the cognitive domain -- over the future 6, 12, 18, and 24 months. We start by training a population-level model using multi-modal data from previously seen subjects using a base Gaussian Process (GP) regression. Then, we personalize this model by adapting the base GP sequentially over time to a new (target) subject using domain adaptive GPs, and also by training subject-specific GP. While we show that these models achieve improved performance when selectively applied to the forecasting task (one performs better than the other on different subjects/visits), the average performance per model is suboptimal. To this end, we used the notion of meta learning in the proposed pGPE to design a regression-based weighting of these expert models, where the expert weights are optimized for each subject and his/her future visit. The results on a cohort of subjects from the ADNI dataset show that this newly introduced personalized weighting of the expert models leads to large improvements in accurately forecasting future ADAS-Cog13 scores and their fine-grained changes associated with the AD progression. This approach has potential to help identify at-risk patients early and improve the construction of clinical trials for AD. △ Less

Submitted 19 April, 2019; originally announced April 2019.

Journal ref: Machine Learning for Healthcare Conference (ML4HC2019)

Deep Learning-Assisted Classification of Site-Resolved Quantum Gas Microscope Images

Authors: Lewis R. B. Picard, Manfred J. Mark, Francesca Ferlaino, Rick van Bijnen

Abstract: We present a novel method for the analysis of quantum gas microscope images, which uses deep learning to improve the fidelity with which lattice sites can be classified as occupied or unoccupied. Our method is especially suited to addressing the case of imaging without continuous cooling, in which the accuracy of existing threshold-based reconstruction methods is limited by atom motion and low pho… ▽ More We present a novel method for the analysis of quantum gas microscope images, which uses deep learning to improve the fidelity with which lattice sites can be classified as occupied or unoccupied. Our method is especially suited to addressing the case of imaging without continuous cooling, in which the accuracy of existing threshold-based reconstruction methods is limited by atom motion and low photon counts. We devise two neural network architectures which are both able to improve upon the fidelity of threshold-based methods, following training on large data sets of simulated images. We evaluate these methods on simulations of a free-space erbium quantum gas microscope, and a noncooled ytterbium microscope in which atoms are pinned in a deep lattice during imaging. In some conditions we see reductions of up to a factor of two in the reconstruction error rate, representing a significant step forward in our efforts to implement high fidelity noncooled site-resolved imaging. △ Less

Submitted 7 November, 2019; v1 submitted 16 April, 2019; originally announced April 2019.

Comments: 18 pages, 10 figures

Journal ref: Meas. Sci. Technol. 31 025201, 2020

Out-of-plane orientation of luminescent excitons in atomically thin indium selenide flakes

Authors: Mauro Brotons-Gisbert, Raphaël Proux, Raphaël Picard, Daniel Andres-Penares, Artur Branny, Alejandro Molina-Sánchez, Juan F. Sánchez-Royo, Brian D. Gerardot

Abstract: Van der Waals materials offer a wide range of atomic layers with unique properties that can be easily combined to engineer novel electronic and photonic devices. A missing ingredient of the van der Waals platform is a two-dimensional crystal with naturally occurring out-of-plane luminescent dipole orientation. Here we measure the far-field photoluminescence intensity distribution of bulk InSe and… ▽ More Van der Waals materials offer a wide range of atomic layers with unique properties that can be easily combined to engineer novel electronic and photonic devices. A missing ingredient of the van der Waals platform is a two-dimensional crystal with naturally occurring out-of-plane luminescent dipole orientation. Here we measure the far-field photoluminescence intensity distribution of bulk InSe and two-dimensional InSe, WSe$_2$ and MoSe$_2$. We demonstrate, with the support of ab-initio calculations, that layered InSe flakes sustain luminescent excitons with an intrinsic out-of-plane orientation, in contrast with the in-plane orientation of dipoles we find in two-dimensional WSe$_2$ and MoSe$_2$ at room-temperature. These results, combined with the high tunability of the optical response and outstanding transport properties, position layered InSe as a promising semiconductor for novel optoelectronic devices, in particular for hybrid integrated photonic chips which exploit the out-of-plane dipole orientation. △ Less

Submitted 26 September, 2019; v1 submitted 20 January, 2019; originally announced January 2019.

Comments: Published in Nature Communications. 12 pages, 5 figures. Link: https://www.nature.com/articles/s41467-019-11920-4

Journal ref: Nature communications 10, 3913 (2019)

Atomically-thin quantum dots integrated with lithium niobate photonic chips

Authors: Daniel White, Artur Branny, Robert J. Chapman, Raphaël Picard, Mauro Brotons-Gisbert, Andreas Boes, Alberto Peruzzo, Cristian Bonato, Brian D. Gerardot

Abstract: The electro-optic, acousto-optic and nonlinear properties of lithium niobate make it a highly versatile material platform for integrated quantum photonic circuits. A prerequisite for quantum technology applications is the ability to efficiently integrate single photon sources, and to guide the generated photons through ad-hoc circuits. Here we report the integration of quantum dots in monolayer WS… ▽ More The electro-optic, acousto-optic and nonlinear properties of lithium niobate make it a highly versatile material platform for integrated quantum photonic circuits. A prerequisite for quantum technology applications is the ability to efficiently integrate single photon sources, and to guide the generated photons through ad-hoc circuits. Here we report the integration of quantum dots in monolayer WSe2 into a Ti in-diffused lithium niobate directional coupler. We investigate the coupling of individual quantum dots to the waveguide mode, their spatial overlap, and the overall efficiency of the hybrid-integrated photonic circuit. △ Less

Submitted 18 October, 2018; originally announced October 2018.

On a Class of Degenerate Abstract Parabolic Problems and Applications to Some Eddy Current Models

Authors: Dirk Pauly, Rainer Picard, Sascha Trostorff, Marcus Waurick

Abstract: We present an abstract framework for parabolic type equations which possibly degenerate on certain spatial regions. The degeneracies are such that the equations under investigation may admit a type change ranging from parabolic to elliptic type problems. The approach is an adaptation of the concept of so-called evolutionary equations in Hilbert spaces and is eventually applied to a degenerate eddy… ▽ More We present an abstract framework for parabolic type equations which possibly degenerate on certain spatial regions. The degeneracies are such that the equations under investigation may admit a type change ranging from parabolic to elliptic type problems. The approach is an adaptation of the concept of so-called evolutionary equations in Hilbert spaces and is eventually applied to a degenerate eddy current type model. The functional analytic setting requires quite minimal assumptions on the boundary and interface regularity. The degenerate eddy current model is justified as a limit model of non-degenerate hyperbolic models of Maxwell's equations. △ Less

Submitted 14 August, 2020; v1 submitted 18 October, 2018; originally announced October 2018.

Comments: 43 pages, additional remarks and references included; referees' comments incorporated

MSC Class: 35Q61; 35M12; 35M32; 35K65; 35K90; 35L90

Coulomb blockade in an atomically thin quantum dot coupled to a tunable Fermi reservoir

Authors: Mauro Brotons-Gisbert, Artur Branny, Santosh Kumar, Raphaël Picard, Raphaël Proux, Mason Gray, Kenneth S. Burch, Kenji Watanabe, Takashi Taniguchi, Brian D. Gerardot

Abstract: Gate-tunable quantum-mechanical tunnelling of particles between a quantum confined state and a nearby Fermi reservoir of delocalized states has underpinned many advances in spintronics and solid-state quantum optics. The prototypical example is a semiconductor quantum dot separated from a gated contact by a tunnel barrier. This enables Coulomb blockade, the phenomenon whereby electrons or holes ca… ▽ More Gate-tunable quantum-mechanical tunnelling of particles between a quantum confined state and a nearby Fermi reservoir of delocalized states has underpinned many advances in spintronics and solid-state quantum optics. The prototypical example is a semiconductor quantum dot separated from a gated contact by a tunnel barrier. This enables Coulomb blockade, the phenomenon whereby electrons or holes can be loaded one-by-one into a quantum dot. Depending on the tunnel-coupling strength, this capability facilitates single spin quantum bits or coherent many-body interactions between the confined spin and the Fermi reservoir. Van der Waals (vdW) heterostructures, in which a wide range of unique atomic layers can easily be combined, offer novel prospects to engineer coherent quantum confined spins, tunnel barriers down to the atomic limit or a Fermi reservoir beyond the conventional flat density of states. However, gate-control of vdW nanostructures at the single particle level is needed to unlock their potential. Here we report Coulomb blockade in a vdW heterostructure consisting of a transition metal dichalcogenide quantum dot coupled to a graphene contact through an atomically thin hexagonal boron nitride (hBN) tunnel barrier. Thanks to a tunable Fermi reservoir, we can deterministically load either a single electron or a single hole into the quantum dot. We observe hybrid excitons, composed of localized quantum dot states and delocalized continuum states, arising from ultra-strong spin-conserving tunnel coupling through the atomically thin tunnel barrier. Probing the charged excitons in applied magnetic fields, we observe large gyromagnetic ratios (~8). Our results establish a foundation for engineering next-generation devices to investigate either novel regimes of Kondo physics or isolated quantum bits in a vdW heterostructure platform. △ Less

Submitted 24 April, 2019; v1 submitted 5 October, 2018; originally announced October 2018.

Comments: Published in Nature Nanotechnology. 7 pages + 14 supplementary information pages. 14 figures

Multi-task multiple kernel machines for personalized pain recognition from functional near-infrared spectroscopy brain signals

Authors: Daniel Lopez-Martinez, Ke Peng, Sarah C. Steele, Arielle J. Lee, David Borsook, Rosalind Picard

Abstract: Currently there is no validated objective measure of pain. Recent neuroimaging studies have explored the feasibility of using functional near-infrared spectroscopy (fNIRS) to measure alterations in brain function in evoked and ongoing pain. In this study, we applied multi-task machine learning methods to derive a practical algorithm for pain detection derived from fNIRS signals in healthy voluntee… ▽ More Currently there is no validated objective measure of pain. Recent neuroimaging studies have explored the feasibility of using functional near-infrared spectroscopy (fNIRS) to measure alterations in brain function in evoked and ongoing pain. In this study, we applied multi-task machine learning methods to derive a practical algorithm for pain detection derived from fNIRS signals in healthy volunteers exposed to a painful stimulus. Especially, we employed multi-task multiple kernel learning to account for the inter-subject variability in pain response. Our results support the use of fNIRS and machine learning techniques in developing objective pain detection, and also highlight the importance of adopting personalized analysis in the process. △ Less

Submitted 21 August, 2018; originally announced August 2018.

Comments: International Conference on Pattern Recognition (ICPR)

A Hilbert space approach to difference equations

Authors: Konrad Kitzing, Rainer Picard, Stefan Siegmund, Sascha Trostorff, Marcus Waurick

Abstract: We consider general difference equations $u_{n+1} = F(u)_n$ for $n \in \mathbb{Z}$ on exponentially weighted $\ell_2$ spaces of two-sided Hilbert space valued sequences $u$ and discuss initial value problems. As an application of the Hilbert space approach, we characterize exponential stability of linear equations and prove a stable manifold theorem for causal nonlinear difference equations. We consider general difference equations $u_{n+1} = F(u)_n$ for $n \in \mathbb{Z}$ on exponentially weighted $\ell_2$ spaces of two-sided Hilbert space valued sequences $u$ and discuss initial value problems. As an application of the Hilbert space approach, we characterize exponential stability of linear equations and prove a stable manifold theorem for causal nonlinear difference equations. △ Less

Submitted 4 October, 2018; v1 submitted 16 July, 2018; originally announced July 2018.

Comments: 23 pages. Reasons for replacement: Added reference, corrected typos

Estimating Carotid Pulse and Breathing Rate from Near-infrared Video of the Neck

Authors: Weixuan Chen, Javier Hernandez, Rosalind W. Picard

Abstract: Objective: Non-contact physiological measurement is a growing research area that allows capturing vital signs such as heart rate (HR) and breathing rate (BR) comfortably and unobtrusively with remote devices. However, most of the approaches work only in bright environments in which subtle photoplethysmographic and ballistocardiographic signals can be easily analyzed and/or require expensive and cu… ▽ More Objective: Non-contact physiological measurement is a growing research area that allows capturing vital signs such as heart rate (HR) and breathing rate (BR) comfortably and unobtrusively with remote devices. However, most of the approaches work only in bright environments in which subtle photoplethysmographic and ballistocardiographic signals can be easily analyzed and/or require expensive and custom hardware to perform the measurements. Approach: This work introduces a low-cost method to measure subtle motions associated with the carotid pulse and breathing movement from the neck using near-infrared (NIR) video imaging. A skin reflection model of the neck was established to provide a theoretical foundation for the method. In particular, the method relies on template matching for neck detection, Principal Component Analysis for feature extraction, and Hidden Markov Models for data smoothing. Main Results: We compared the estimated HR and BR measures with ones provided by an FDA-cleared device in a 12-participant laboratory study: the estimates achieved a mean absolute error of 0.36 beats per minute and 0.24 breaths per minute under both bright and dark lighting. Significance: This work advances the possibilities of non-contact physiological measurement in real-life conditions in which environmental illumination is limited and in which the face of the person is not readily available or needs to be protected. Due to the increasing availability of NIR imaging devices, the described methods are readily scalable. △ Less

Submitted 24 May, 2018; originally announced May 2018.

Comments: 21 pages, 15 figures