-
ReXamine-Global: A Framework for Uncovering Inconsistencies in Radiology Report Generation Metrics
Authors:
Oishi Banerjee,
Agustina Saenz,
Kay Wu,
Warren Clements,
Adil Zia,
Dominic Buensalido,
Helen Kavnoudias,
Alain S. Abi-Ghanem,
Nour El Ghawi,
Cibele Luna,
Patricia Castillo,
Khaled Al-Surimi,
Rayyan A. Daghistani,
Yuh-Min Chen,
Heng-sheng Chao,
Lars Heiliger,
Moon Kim,
Johannes Haubold,
Frederic Jonske,
Pranav Rajpurkar
Abstract:
Given the rapidly expanding capabilities of generative AI models for radiology, there is a need for robust metrics that can accurately measure the quality of AI-generated radiology reports across diverse hospitals. We develop ReXamine-Global, a LLM-powered, multi-site framework that tests metrics across different writing styles and patient populations, exposing gaps in their generalization. First,…
▽ More
Given the rapidly expanding capabilities of generative AI models for radiology, there is a need for robust metrics that can accurately measure the quality of AI-generated radiology reports across diverse hospitals. We develop ReXamine-Global, a LLM-powered, multi-site framework that tests metrics across different writing styles and patient populations, exposing gaps in their generalization. First, our method tests whether a metric is undesirably sensitive to reporting style, providing different scores depending on whether AI-generated reports are stylistically similar to ground-truth reports or not. Second, our method measures whether a metric reliably agrees with experts, or whether metric and expert scores of AI-generated report quality diverge for some sites. Using 240 reports from 6 hospitals around the world, we apply ReXamine-Global to 7 established report evaluation metrics and uncover serious gaps in their generalizability. Developers can apply ReXamine-Global when designing new report evaluation metrics, ensuring their robustness across sites. Additionally, our analysis of existing metrics can guide users of those metrics towards evaluation procedures that work reliably at their sites of interest.
△ Less
Submitted 28 August, 2024;
originally announced August 2024.
-
Magicity versus superfluidity around $^{28}$O viewed from the study of $^{30}$F
Authors:
J. Kahlbow,
T. Aumann,
O. Sorlin,
Y. Kondo,
T. Nakamura,
F. Nowacki,
A. Revel,
N. L. Achouri,
H. Al Falou,
L. Atar,
H. Baba,
K. Boretzky,
C. Caesar,
D. Calvet,
H. Chae,
N. Chiga,
A. Corsi,
F. Delaunay,
A. Delbart,
Q. Deshayes,
Z. Dombradi,
C. A. Douma,
Z. Elekes,
I. Gasparic,
J. -M. Gheller
, et al. (62 additional authors not shown)
Abstract:
The neutron-rich unbound fluorine isotope $^{30}$F$_{21}$ has been observed for the first time by measuring its neutron decay at the SAMURAI spectrometer (RIBF, RIKEN) in the quasi-free proton knockout reaction of $^{31}$Ne nuclei at 235 MeV/nucleon. The mass and thus one-neutron-separation energy of $^{30}$F has been determined to be $S_n = -472\pm 58 \mathrm{(stat.)} \pm 33 \mathrm{(sys.)}$ keV…
▽ More
The neutron-rich unbound fluorine isotope $^{30}$F$_{21}$ has been observed for the first time by measuring its neutron decay at the SAMURAI spectrometer (RIBF, RIKEN) in the quasi-free proton knockout reaction of $^{31}$Ne nuclei at 235 MeV/nucleon. The mass and thus one-neutron-separation energy of $^{30}$F has been determined to be $S_n = -472\pm 58 \mathrm{(stat.)} \pm 33 \mathrm{(sys.)}$ keV from the measurement of its invariant-mass spectrum. The absence of a sharp drop in $S_n$($^{30}$F) shows that the ``magic'' $N=20$ shell gap is not restored close to $^{28}$O, which is in agreement with our shell-model calculations that predict a near degeneracy between the neutron $d$ and $fp$ orbitals, with the $1p_{3/2}$ and $1p_{1/2}$ orbitals becoming more bound than the $0f_{7/2}$ one. This degeneracy and reordering of orbitals has two potential consequences: $^{28}$O behaves like a strongly superfluid nucleus with neutron pairs scattering across shells, and both $^{29,31}$F appear to be good two-neutron halo-nucleus candidates.
△ Less
Submitted 27 July, 2024;
originally announced July 2024.
-
DreamStory: Open-Domain Story Visualization by LLM-Guided Multi-Subject Consistent Diffusion
Authors:
Huiguo He,
Huan Yang,
Zixi Tuo,
Yuan Zhou,
Qiuyue Wang,
Yuhang Zhang,
Zeyu Liu,
Wenhao Huang,
Hongyang Chao,
Jian Yin
Abstract:
Story visualization aims to create visually compelling images or videos corresponding to textual narratives. Despite recent advances in diffusion models yielding promising results, existing methods still struggle to create a coherent sequence of subject-consistent frames based solely on a story. To this end, we propose DreamStory, an automatic open-domain story visualization framework by leveragin…
▽ More
Story visualization aims to create visually compelling images or videos corresponding to textual narratives. Despite recent advances in diffusion models yielding promising results, existing methods still struggle to create a coherent sequence of subject-consistent frames based solely on a story. To this end, we propose DreamStory, an automatic open-domain story visualization framework by leveraging the LLMs and a novel multi-subject consistent diffusion model. DreamStory consists of (1) an LLM acting as a story director and (2) an innovative Multi-Subject consistent Diffusion model (MSD) for generating consistent multi-subject across the images. First, DreamStory employs the LLM to generate descriptive prompts for subjects and scenes aligned with the story, annotating each scene's subjects for subsequent subject-consistent generation. Second, DreamStory utilizes these detailed subject descriptions to create portraits of the subjects, with these portraits and their corresponding textual information serving as multimodal anchors (guidance). Finally, the MSD uses these multimodal anchors to generate story scenes with consistent multi-subject. Specifically, the MSD includes Masked Mutual Self-Attention (MMSA) and Masked Mutual Cross-Attention (MMCA) modules. MMSA and MMCA modules ensure appearance and semantic consistency with reference images and text, respectively. Both modules employ masking mechanisms to prevent subject blending. To validate our approach and promote progress in story visualization, we established a benchmark, DS-500, which can assess the overall performance of the story visualization framework, subject-identification accuracy, and the consistency of the generation model. Extensive experiments validate the effectiveness of DreamStory in both subjective and objective evaluations. Please visit our project homepage at https://dream-xyz.github.io/dreamstory.
△ Less
Submitted 17 July, 2024;
originally announced July 2024.
-
Do LLMs Have Distinct and Consistent Personality? TRAIT: Personality Testset designed for LLMs with Psychometrics
Authors:
Seungbeen Lee,
Seungwon Lim,
Seungju Han,
Giyeong Oh,
Hyungjoo Chae,
Jiwan Chung,
Minju Kim,
Beong-woo Kwak,
Yeonsoo Lee,
Dongha Lee,
Jinyoung Yeo,
Youngjae Yu
Abstract:
The idea of personality in descriptive psychology, traditionally defined through observable behavior, has now been extended to Large Language Models (LLMs) to better understand their behavior. This raises a question: do LLMs exhibit distinct and consistent personality traits, similar to humans? Existing self-assessment personality tests, while applicable, lack the necessary validity and reliabilit…
▽ More
The idea of personality in descriptive psychology, traditionally defined through observable behavior, has now been extended to Large Language Models (LLMs) to better understand their behavior. This raises a question: do LLMs exhibit distinct and consistent personality traits, similar to humans? Existing self-assessment personality tests, while applicable, lack the necessary validity and reliability for precise personality measurements. To address this, we introduce TRAIT, a new tool consisting of 8K multi-choice questions designed to assess the personality of LLMs with validity and reliability. TRAIT is built on the psychometrically validated human questionnaire, Big Five Inventory (BFI) and Short Dark Triad (SD-3), enhanced with the ATOMIC10X knowledge graph for testing personality in a variety of real scenarios. TRAIT overcomes the reliability and validity issues when measuring personality of LLM with self-assessment, showing the highest scores across three metrics: refusal rate, prompt sensitivity, and option order sensitivity. It reveals notable insights into personality of LLM: 1) LLMs exhibit distinct and consistent personality, which is highly influenced by their training data (i.e., data used for alignment tuning), and 2) current prompting techniques have limited effectiveness in eliciting certain traits, such as high psychopathy or low conscientiousness, suggesting the need for further research in this direction.
△ Less
Submitted 20 June, 2024;
originally announced June 2024.
-
The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models
Authors:
Seungone Kim,
Juyoung Suk,
Ji Yong Cho,
Shayne Longpre,
Chaeeun Kim,
Dongkeun Yoon,
Guijin Son,
Yejin Cho,
Sheikh Shafayat,
Jinheon Baek,
Sue Hyun Park,
Hyeonbin Hwang,
Jinkyung Jo,
Hyowon Cho,
Haebin Shin,
Seongyun Lee,
Hanseok Oh,
Noah Lee,
Namgyu Ho,
Se June Joo,
Miyoung Ko,
Yoonjoo Lee,
Hyungjoo Chae,
Jamin Shin,
Joel Jang
, et al. (7 additional authors not shown)
Abstract:
As language models (LMs) become capable of handling a wide range of tasks, their evaluation is becoming as challenging as their development. Most generation benchmarks currently assess LMs using abstract evaluation criteria like helpfulness and harmlessness, which often lack the flexibility and granularity of human assessment. Additionally, these benchmarks tend to focus disproportionately on spec…
▽ More
As language models (LMs) become capable of handling a wide range of tasks, their evaluation is becoming as challenging as their development. Most generation benchmarks currently assess LMs using abstract evaluation criteria like helpfulness and harmlessness, which often lack the flexibility and granularity of human assessment. Additionally, these benchmarks tend to focus disproportionately on specific capabilities such as instruction following, leading to coverage bias. To overcome these limitations, we introduce the BiGGen Bench, a principled generation benchmark designed to thoroughly evaluate nine distinct capabilities of LMs across 77 diverse tasks. A key feature of the BiGGen Bench is its use of instance-specific evaluation criteria, closely mirroring the nuanced discernment of human evaluation. We apply this benchmark to assess 103 frontier LMs using five evaluator LMs. Our code, data, and evaluation results are all publicly available at https://github.com/prometheus-eval/prometheus-eval/tree/main/BiGGen-Bench.
△ Less
Submitted 9 June, 2024;
originally announced June 2024.
-
Geospatial Big Data: Survey and Challenges
Authors:
Jiayang Wu,
Wensheng Gan,
Han-Chieh Chao,
Philip S. Yu
Abstract:
In recent years, geospatial big data (GBD) has obtained attention across various disciplines, categorized into big earth observation data and big human behavior data. Identifying geospatial patterns from GBD has been a vital research focus in the fields of urban management and environmental sustainability. This paper reviews the evolution of GBD mining and its integration with advanced artificial…
▽ More
In recent years, geospatial big data (GBD) has obtained attention across various disciplines, categorized into big earth observation data and big human behavior data. Identifying geospatial patterns from GBD has been a vital research focus in the fields of urban management and environmental sustainability. This paper reviews the evolution of GBD mining and its integration with advanced artificial intelligence (AI) techniques. GBD consists of data generated by satellites, sensors, mobile devices, and geographical information systems, and we categorize geospatial data based on different perspectives. We outline the process of GBD mining and demonstrate how it can be incorporated into a unified framework. Additionally, we explore new technologies like large language models (LLM), the Metaverse, and knowledge graphs, and how they could make GBD even more useful. We also share examples of GBD helping with city management and protecting the environment. Finally, we discuss the real challenges that come up when working with GBD, such as issues with data retrieval and security. Our goal is to give readers a clear view of where GBD mining stands today and where it might go next.
△ Less
Submitted 29 April, 2024;
originally announced April 2024.
-
A self-consistent Hartree theory for lattice-relaxed magic-angle twisted bilayer graphene
Authors:
Mohammed M. Al Ezzi,
Liangtao Peng,
Zhengyu Liu,
Jonah Huang Zi Chao,
Gayani N. Pallewela,
Darryl Foo,
Shaffique Adam
Abstract:
For twisted bilayer graphene close to magic angle, we show that the effects of lattice relaxation and the Hartree interaction both become simultaneously important. Including both effects in a continuum theory reveals a Lifshitz transition to a Fermi surface topology that supports both a ``heavy fermion" pocket and an ultraflat band ($\approx 8~{\rm meV}$) that is pinned to the Fermi energy for a l…
▽ More
For twisted bilayer graphene close to magic angle, we show that the effects of lattice relaxation and the Hartree interaction both become simultaneously important. Including both effects in a continuum theory reveals a Lifshitz transition to a Fermi surface topology that supports both a ``heavy fermion" pocket and an ultraflat band ($\approx 8~{\rm meV}$) that is pinned to the Fermi energy for a large range of fillings. We provide analytical and numerical results to understand the narrow ``magic angle range" that supports this pinned ultraflat band and make predictions for its experimental observation. We believe that the bands presented here are accurate at high temperature and provide a good starting point to understand the myriad of complex behaviour observed in this system.
△ Less
Submitted 26 April, 2024;
originally announced April 2024.
-
On the number of subsequence sums related to the support of a sequence in finite abelian groups
Authors:
Rui Wang,
Han Chao,
Jiangtao Peng
Abstract:
Let $G$ be a finite abelian group and $S$ a sequence with elements of $G$. Let $|S|$ denote the length of $S$ and $\mathrm{supp}(S)$ the set of all the distinct terms in $S$. For an integer $k$ with $k\in [1, |S|]$, let $Σ_{k}(S) \subset G$ denote the set of group elements which can be expressed as a sum of a subsequence of $S$ with length $k$. Let $Σ(S)=\cup_{k=1}^{|S|}Σ_{k}(S)$ and…
▽ More
Let $G$ be a finite abelian group and $S$ a sequence with elements of $G$. Let $|S|$ denote the length of $S$ and $\mathrm{supp}(S)$ the set of all the distinct terms in $S$. For an integer $k$ with $k\in [1, |S|]$, let $Σ_{k}(S) \subset G$ denote the set of group elements which can be expressed as a sum of a subsequence of $S$ with length $k$. Let $Σ(S)=\cup_{k=1}^{|S|}Σ_{k}(S)$ and $Σ_{\geq k}(S)=\cup_{t=k}^{|S|}Σ_{t}(S)$. It is known that if $0\not\in Σ(S)$, then $|Σ(S)|\geq |S|+|\mathrm{supp}(S)|-1$. In this paper, we determine the structure of a sequence $S$ satisfying $0\notin Σ(S)$ and $|Σ(S)|= |S|+|\mathrm{supp}(S)|-1$. As a consequence, we can give a counterexample of a conjecture of Gao, Grynkiewicz, and Xia. Moreover, we prove that if $|S|>k$ and $0\not\in Σ_{\geq k}(S)\cup \mathrm{supp}(S)$, then $|Σ_{\geq k}(S)|\geq |S|-k+|\mathrm{supp}(S)|$. Then we can give an alternative proof of a conjecture of Hamidoune, which was first proved by Gao, Grynkiewicz, and Xia.
△ Less
Submitted 29 April, 2024; v1 submitted 17 April, 2024;
originally announced April 2024.
-
Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models
Authors:
Hyungjoo Chae,
Yeonghyeon Kim,
Seungone Kim,
Kai Tzu-iunn Ong,
Beong-woo Kwak,
Moohyeon Kim,
Seonghwan Kim,
Taeyoon Kwon,
Jiwan Chung,
Youngjae Yu,
Jinyoung Yeo
Abstract:
Algorithmic reasoning refers to the ability to understand the complex patterns behind the problem and decompose them into a sequence of reasoning steps towards the solution. Such nature of algorithmic reasoning makes it a challenge for large language models (LLMs), even though they have demonstrated promising performance in other reasoning tasks. Within this context, some recent studies use progra…
▽ More
Algorithmic reasoning refers to the ability to understand the complex patterns behind the problem and decompose them into a sequence of reasoning steps towards the solution. Such nature of algorithmic reasoning makes it a challenge for large language models (LLMs), even though they have demonstrated promising performance in other reasoning tasks. Within this context, some recent studies use programming languages (e.g., Python) to express the necessary logic for solving a given instance/question (e.g., Program-of-Thought) as inspired by their strict and precise syntaxes. However, it is non-trivial to write an executable code that expresses the correct logic on the fly within a single inference call. Also, the code generated specifically for an instance cannot be reused for others, even if they are from the same task and might require identical logic to solve. This paper presents Think-and-Execute, a novel framework that decomposes the reasoning process of language models into two steps. (1) In Think, we discover a task-level logic that is shared across all instances for solving a given task and then express the logic with pseudocode; (2) In Execute, we further tailor the generated pseudocode to each instance and simulate the execution of the code. With extensive experiments on seven algorithmic reasoning tasks, we demonstrate the effectiveness of Think-and-Execute. Our approach better improves LMs' reasoning compared to several strong baselines performing instance-specific reasoning (e.g., CoT and PoT), suggesting the helpfulness of discovering task-level logic. Also, we show that compared to natural language, pseudocode can better guide the reasoning of LMs, even though they are trained to follow natural language instructions.
△ Less
Submitted 3 April, 2024;
originally announced April 2024.
-
Enhancing Ship Classification in Optical Satellite Imagery: Integrating Convolutional Block Attention Module with ResNet for Improved Performance
Authors:
Ryan Donghan Kwon,
Gangjoo Robin Nam,
Jisoo Tak,
Junseob Shin,
Hyerin Cha,
Seung Won Lee
Abstract:
In this study, we present an advanced convolutional neural network (CNN) architecture for ship classification based on optical satellite imagery, which significantly enhances performance through the integration of a convolutional block attention module (CBAM) and additional architectural innovations. Building upon the foundational ResNet50 model, we first incorporated a standard CBAM to direct the…
▽ More
In this study, we present an advanced convolutional neural network (CNN) architecture for ship classification based on optical satellite imagery, which significantly enhances performance through the integration of a convolutional block attention module (CBAM) and additional architectural innovations. Building upon the foundational ResNet50 model, we first incorporated a standard CBAM to direct the model's focus toward more informative features, achieving an accuracy of 87% compared to 85% of the baseline ResNet50. Further augmentations involved multiscale feature integration, depthwise separable convolutions, and dilated convolutions, culminating in an enhanced ResNet model with improved CBAM. This model demonstrated a remarkable accuracy of 95%, with precision, recall, and F1 scores all witnessing substantial improvements across various ship classes. In particular, the bulk carrier and oil tanker classes exhibited nearly perfect precision and recall rates, underscoring the enhanced capability of the model to accurately identify and classify ships. Attention heatmap analyses further validated the efficacy of the improved model, revealing more focused attention on relevant ship features regardless of background complexities. These findings underscore the potential of integrating attention mechanisms and architectural innovations into CNNs for high-resolution satellite imagery classification. This study navigates through the class imbalance and computational costs and proposes future directions for scalability and adaptability in new or rare ship-type recognition. This study lays the groundwork for applying advanced deep learning techniques in remote sensing, offering insights into scalable and efficient satellite image classification.
△ Less
Submitted 20 August, 2024; v1 submitted 2 April, 2024;
originally announced April 2024.
-
Evidence-Focused Fact Summarization for Knowledge-Augmented Zero-Shot Question Answering
Authors:
Sungho Ko,
Hyunjin Cho,
Hyungjoo Chae,
Jinyoung Yeo,
Dongha Lee
Abstract:
Recent studies have investigated utilizing Knowledge Graphs (KGs) to enhance Quesetion Answering (QA) performance of Large Language Models (LLMs), yet structured KG verbalization remains challengin. Existing methods, such as triple-form or free-form textual conversion of triple-form facts, encounter several issues. These include reduced evidence density due to duplicated entities or relationships,…
▽ More
Recent studies have investigated utilizing Knowledge Graphs (KGs) to enhance Quesetion Answering (QA) performance of Large Language Models (LLMs), yet structured KG verbalization remains challengin. Existing methods, such as triple-form or free-form textual conversion of triple-form facts, encounter several issues. These include reduced evidence density due to duplicated entities or relationships, and reduced evidence clarity due to an inability to emphasize crucial evidence. To address these issues, we propose EFSum, an Evidence-focused Fact Summarization framework for enhanced QA with knowledge-augmented LLMs. We optimize an open-source LLM as a fact summarizer through distillation and preference alignment. Our extensive experiments show that EFSum improves LLM's zero-shot QA performance, and it is possible to ensure both the helpfulness and faithfulness of the summary.
△ Less
Submitted 19 June, 2024; v1 submitted 5 March, 2024;
originally announced March 2024.
-
Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers
Authors:
Tsai-Shien Chen,
Aliaksandr Siarohin,
Willi Menapace,
Ekaterina Deyneka,
Hsiang-wei Chao,
Byung Eun Jeon,
Yuwei Fang,
Hsin-Ying Lee,
Jian Ren,
Ming-Hsuan Yang,
Sergey Tulyakov
Abstract:
The quality of the data and annotation upper-bounds the quality of a downstream model. While there exist large text corpora and image-text pairs, high-quality video-text data is much harder to collect. First of all, manual labeling is more time-consuming, as it requires an annotator to watch an entire video. Second, videos have a temporal dimension, consisting of several scenes stacked together, a…
▽ More
The quality of the data and annotation upper-bounds the quality of a downstream model. While there exist large text corpora and image-text pairs, high-quality video-text data is much harder to collect. First of all, manual labeling is more time-consuming, as it requires an annotator to watch an entire video. Second, videos have a temporal dimension, consisting of several scenes stacked together, and showing multiple actions. Accordingly, to establish a video dataset with high-quality captions, we propose an automatic approach leveraging multimodal inputs, such as textual video description, subtitles, and individual video frames. Specifically, we curate 3.8M high-resolution videos from the publicly available HD-VILA-100M dataset. We then split them into semantically consistent video clips, and apply multiple cross-modality teacher models to obtain captions for each video. Next, we finetune a retrieval model on a small subset where the best caption of each video is manually selected and then employ the model in the whole dataset to select the best caption as the annotation. In this way, we get 70M videos paired with high-quality text captions. We dub the dataset as Panda-70M. We show the value of the proposed dataset on three downstream tasks: video captioning, video and text retrieval, and text-driven video generation. The models trained on the proposed data score substantially better on the majority of metrics across all the tasks.
△ Less
Submitted 29 February, 2024;
originally announced February 2024.
-
VerifiNER: Verification-augmented NER via Knowledge-grounded Reasoning with Large Language Models
Authors:
Seoyeon Kim,
Kwangwook Seo,
Hyungjoo Chae,
Jinyoung Yeo,
Dongha Lee
Abstract:
Recent approaches in domain-specific named entity recognition (NER), such as biomedical NER, have shown remarkable advances. However, they still lack of faithfulness, producing erroneous predictions. We assume that knowledge of entities can be useful in verifying the correctness of the predictions. Despite the usefulness of knowledge, resolving such errors with knowledge is nontrivial, since the k…
▽ More
Recent approaches in domain-specific named entity recognition (NER), such as biomedical NER, have shown remarkable advances. However, they still lack of faithfulness, producing erroneous predictions. We assume that knowledge of entities can be useful in verifying the correctness of the predictions. Despite the usefulness of knowledge, resolving such errors with knowledge is nontrivial, since the knowledge itself does not directly indicate the ground-truth label. To this end, we propose VerifiNER, a post-hoc verification framework that identifies errors from existing NER methods using knowledge and revises them into more faithful predictions. Our framework leverages the reasoning abilities of large language models to adequately ground on knowledge and the contextual information in the verification process. We validate effectiveness of VerifiNER through extensive experiments on biomedical datasets. The results suggest that VerifiNER can successfully verify errors from existing models as a model-agnostic approach. Further analyses on out-of-domain and low-resource settings show the usefulness of VerifiNER on real-world applications.
△ Less
Submitted 8 June, 2024; v1 submitted 28 February, 2024;
originally announced February 2024.
-
PEGASUS: Personalized Generative 3D Avatars with Composable Attributes
Authors:
Hyunsoo Cha,
Byungjun Kim,
Hanbyul Joo
Abstract:
We present PEGASUS, a method for constructing a personalized generative 3D face avatar from monocular video sources. Our generative 3D avatar enables disentangled controls to selectively alter the facial attributes (e.g., hair or nose) while preserving the identity. Our approach consists of two stages: synthetic database generation and constructing a personalized generative avatar. We generate a s…
▽ More
We present PEGASUS, a method for constructing a personalized generative 3D face avatar from monocular video sources. Our generative 3D avatar enables disentangled controls to selectively alter the facial attributes (e.g., hair or nose) while preserving the identity. Our approach consists of two stages: synthetic database generation and constructing a personalized generative avatar. We generate a synthetic video collection of the target identity with varying facial attributes, where the videos are synthesized by borrowing the attributes from monocular videos of diverse identities. Then, we build a person-specific generative 3D avatar that can modify its attributes continuously while preserving its identity. Through extensive experiments, we demonstrate that our method of generating a synthetic database and creating a 3D generative avatar is the most effective in preserving identity while achieving high realism. Subsequently, we introduce a zero-shot approach to achieve the same goal of generative modeling more efficiently by leveraging a previously constructed personalized generative model.
△ Less
Submitted 2 April, 2024; v1 submitted 16 February, 2024;
originally announced February 2024.
-
Multimodal Neurodegenerative Disease Subtyping Explained by ChatGPT
Authors:
Diego Machado Reyes,
Hanqing Chao,
Juergen Hahn,
Li Shen,
Pingkun Yan
Abstract:
Alzheimer's disease (AD) is the most prevalent neurodegenerative disease; yet its currently available treatments are limited to stopping disease progression. Moreover, effectiveness of these treatments is not guaranteed due to the heterogenetiy of the disease. Therefore, it is essential to be able to identify the disease subtypes at a very early stage. Current data driven approaches are able to cl…
▽ More
Alzheimer's disease (AD) is the most prevalent neurodegenerative disease; yet its currently available treatments are limited to stopping disease progression. Moreover, effectiveness of these treatments is not guaranteed due to the heterogenetiy of the disease. Therefore, it is essential to be able to identify the disease subtypes at a very early stage. Current data driven approaches are able to classify the subtypes at later stages of AD or related disorders, but struggle when predicting at the asymptomatic or prodromal stage. Moreover, most existing models either lack explainability behind the classification or only use a single modality for the assessment, limiting scope of its analysis. Thus, we propose a multimodal framework that uses early-stage indicators such as imaging, genetics and clinical assessments to classify AD patients into subtypes at early stages. Similarly, we build prompts and use large language models, such as ChatGPT, to interpret the findings of our model. In our framework, we propose a tri-modal co-attention mechanism (Tri-COAT) to explicitly learn the cross-modal feature associations. Our proposed model outperforms baseline models and provides insight into key cross-modal feature associations supported by known biological mechanisms.
△ Less
Submitted 31 January, 2024;
originally announced February 2024.
-
5G NR Positioning Enhancements in 3GPP Release-18
Authors:
Hyun-Su Cha,
Gilsoo Lee,
Amitava Ghosh,
Matthew Baker,
Sean Kelley,
Juergen Hofmann
Abstract:
New radio (NR) positioning in the Third Generation Partnership Project (3GPP) Release 18 (Rel-18) enables 5G-advanced networks to achieve ultra-high accuracy positioning without dependence on global navigation satellite systems (GNSS) with key enablers such as the carrier phase positioning technique, standardized for the first time in a cellular communications standard and setting a new baseline f…
▽ More
New radio (NR) positioning in the Third Generation Partnership Project (3GPP) Release 18 (Rel-18) enables 5G-advanced networks to achieve ultra-high accuracy positioning without dependence on global navigation satellite systems (GNSS) with key enablers such as the carrier phase positioning technique, standardized for the first time in a cellular communications standard and setting a new baseline for future generations. In addition, Rel-18 NR supports positioning functionalities for reduced capability (RedCap) user equipment and bandwidth aggregation for positioning measurements. Moreover, the low power solutions are designed for low power high accuracy positioning use cases. Lastly, sidelink-based positioning is introduced in Rel-18. This article constitutes a comprehensive treatment of the Rel-18 NR positioning enhancements crucial for the development of next-generation networks.
△ Less
Submitted 30 January, 2024;
originally announced January 2024.
-
Coffee: Boost Your Code LLMs by Fixing Bugs with Feedback
Authors:
Seungjun Moon,
Hyungjoo Chae,
Yongho Song,
Taeyoon Kwon,
Dongjin Kang,
Kai Tzu-iunn Ong,
Seung-won Hwang,
Jinyoung Yeo
Abstract:
Code editing is an essential step towards reliable program synthesis to automatically correct critical errors generated from code LLMs. Recent studies have demonstrated that closed-source LLMs (i.e., ChatGPT and GPT-4) are capable of generating corrective feedback to edit erroneous inputs. However, it remains challenging for open-source code LLMs to generate feedback for code editing, since these…
▽ More
Code editing is an essential step towards reliable program synthesis to automatically correct critical errors generated from code LLMs. Recent studies have demonstrated that closed-source LLMs (i.e., ChatGPT and GPT-4) are capable of generating corrective feedback to edit erroneous inputs. However, it remains challenging for open-source code LLMs to generate feedback for code editing, since these models tend to adhere to the superficial formats of feedback and provide feedback with misleading information. Hence, the focus of our work is to leverage open-source code LLMs to generate helpful feedback with correct guidance for code editing. To this end, we present Coffee, a collected dataset specifically designed for code fixing with feedback. Using this dataset, we construct CoffeePots, a framework for COde Fixing with FEEdback via Preference-Optimized Tuning and Selection. The proposed framework aims to automatically generate helpful feedback for code editing while minimizing the potential risk of superficial feedback. The combination of Coffee and CoffeePots marks a significant advancement, achieving state-of-the-art performance on HumanEvalFix benchmark. Codes and model checkpoints are publicly available at https://github.com/Lune-Blue/COFFEE.
△ Less
Submitted 23 February, 2024; v1 submitted 13 November, 2023;
originally announced November 2023.
-
An infrared, Raman, and X-ray database of battery interphase components
Authors:
Lukas Karapin-Springorum,
Asia Sarycheva,
Andrew Dopilka,
Hyungyeon Cha,
Muhammad Ihsan-Ul-Haq,
Jonathan M. Larson,
Robert Kostecki
Abstract:
Further technological advancement of both lithium-ion and emerging battery technologies can be catalyzed by an improved understanding of the chemistry and working mechanisms of the solid electrolyte interphases (SEIs) that form at electrochemically active battery interfaces. However, collecting and interpreting spectroscopy results of SEIs is difficult for several reasons, including the chemically…
▽ More
Further technological advancement of both lithium-ion and emerging battery technologies can be catalyzed by an improved understanding of the chemistry and working mechanisms of the solid electrolyte interphases (SEIs) that form at electrochemically active battery interfaces. However, collecting and interpreting spectroscopy results of SEIs is difficult for several reasons, including the chemically diverse composition of SEIs. To address this challenge, we herein present a vibrational spectroscopy and X-ray diffraction data library of ten suggested SEI chemical constituents relevant to both lithium-ion and emerging battery chemistries. The data library includes attenuated total reflectance Fourier transform infrared spectroscopy, Raman spectroscopy, and X-ray diffraction data, collected in inert atmospheres afforded by custom designed sample holders. The data library presented in this work (and online repository) alleviates challenges with locating related work that is either diffusely spread throughout the literature, or is non-existent, and provides energy storage researchers streamlined access to vital SEI-relevant data that can catalyse future battery research efforts.
△ Less
Submitted 20 October, 2023;
originally announced October 2023.
-
Fabrication of electronically conductive protein-heme nanowires for power harvesting
Authors:
Lorenzo Travaglini,
Nga T. Lam,
Artur Sawicki,
Hee-Jeong Cha,
Dawei Xu,
Adam P. Micolich,
Douglas S. Clark,
Dominic J. Glover
Abstract:
Electronically conductive protein-based materials could enable the creation of bioelectronic components and devices from sustainable and nontoxic materials, while also being well-suited to interface with biological systems, such as living cells, for biosensor applications. In addition, protein materials have other desirable properties such as inherent self-assembly and molecular recognition capabi…
▽ More
Electronically conductive protein-based materials could enable the creation of bioelectronic components and devices from sustainable and nontoxic materials, while also being well-suited to interface with biological systems, such as living cells, for biosensor applications. In addition, protein materials have other desirable properties such as inherent self-assembly and molecular recognition capabilities. However, as proteins are generally electrical insulators, the ability to render protein assemblies electronically conductive in a tailorable manner could usher in a plethora of useful materials. Here, we present an approach to fabricate electronically conductive protein nanowires by incorporating and aligning heme molecules in proximity along an ultrastable protein filament. The heme-incorporated protein nanowires demonstrated electron transfer over micrometer distances, with conductive atomic force microscopy showing individual nanowires having comparable conductance to naturally occurring bacterial nanowires. The heme-incorporated nanowires were also capable of harvesting energy from ambient humidity when deposited as multilayer films. Exposure of films to humidity produced electrical current, presumably through water molecules ionizing carboxy groups in the protein filament and creating an unbalanced total charge distribution that is enhanced by the presence of heme. A wide variety of other porphyrin molecules exist with varying electrochemical behaviors that could enable the electrical properties of protein assemblies to be tailored, paving the way to structurally- and electrically-defined protein-based bioelectronic devices.
△ Less
Submitted 15 October, 2023;
originally announced October 2023.
-
Dialogue Chain-of-Thought Distillation for Commonsense-aware Conversational Agents
Authors:
Hyungjoo Chae,
Yongho Song,
Kai Tzu-iunn Ong,
Taeyoon Kwon,
Minjin Kim,
Youngjae Yu,
Dongha Lee,
Dongyeop Kang,
Jinyoung Yeo
Abstract:
Human-like chatbots necessitate the use of commonsense reasoning in order to effectively comprehend and respond to implicit information present within conversations. Achieving such coherence and informativeness in responses, however, is a non-trivial task. Even for large language models (LLMs), the task of identifying and aggregating key evidence within a single hop presents a substantial challeng…
▽ More
Human-like chatbots necessitate the use of commonsense reasoning in order to effectively comprehend and respond to implicit information present within conversations. Achieving such coherence and informativeness in responses, however, is a non-trivial task. Even for large language models (LLMs), the task of identifying and aggregating key evidence within a single hop presents a substantial challenge. This complexity arises because such evidence is scattered across multiple turns in a conversation, thus necessitating integration over multiple hops. Hence, our focus is to facilitate such multi-hop reasoning over a dialogue context, namely dialogue chain-of-thought (CoT) reasoning. To this end, we propose a knowledge distillation framework that leverages LLMs as unreliable teachers and selectively distills consistent and helpful rationales via alignment filters. We further present DOCTOR, a DialOgue Chain-of-ThOught Reasoner that provides reliable CoT rationales for response generation. We conduct extensive experiments to show that enhancing dialogue agents with high-quality rationales from DOCTOR significantly improves the quality of their responses.
△ Less
Submitted 22 October, 2023; v1 submitted 13 October, 2023;
originally announced October 2023.
-
TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance
Authors:
Kan Wu,
Houwen Peng,
Zhenghong Zhou,
Bin Xiao,
Mengchen Liu,
Lu Yuan,
Hong Xuan,
Michael Valenzuela,
Xi,
Chen,
Xinggang Wang,
Hongyang Chao,
Han Hu
Abstract:
In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models. The method introduces two core techniques: affinity mimicking and weight inheritance. Affinity mimicking explores the interaction between modalities during distillation, enabling student models to mimic teachers' behavior of learning cross-modal feature alignment i…
▽ More
In this paper, we propose a novel cross-modal distillation method, called TinyCLIP, for large-scale language-image pre-trained models. The method introduces two core techniques: affinity mimicking and weight inheritance. Affinity mimicking explores the interaction between modalities during distillation, enabling student models to mimic teachers' behavior of learning cross-modal feature alignment in a visual-linguistic affinity space. Weight inheritance transmits the pre-trained weights from the teacher models to their student counterparts to improve distillation efficiency. Moreover, we extend the method into a multi-stage progressive distillation to mitigate the loss of informative weights during extreme compression. Comprehensive experiments demonstrate the efficacy of TinyCLIP, showing that it can reduce the size of the pre-trained CLIP ViT-B/32 by 50%, while maintaining comparable zero-shot performance. While aiming for comparable performance, distillation with weight inheritance can speed up the training by 1.4 - 7.8 $\times$ compared to training from scratch. Moreover, our TinyCLIP ViT-8M/16, trained on YFCC-15M, achieves an impressive zero-shot top-1 accuracy of 41.1% on ImageNet, surpassing the original CLIP ViT-B/16 by 3.5% while utilizing only 8.9% parameters. Finally, we demonstrate the good transferability of TinyCLIP in various downstream tasks. Code and models will be open-sourced at https://aka.ms/tinyclip.
△ Less
Submitted 21 September, 2023;
originally announced September 2023.
-
Spectral Adversarial MixUp for Few-Shot Unsupervised Domain Adaptation
Authors:
Jiajin Zhang,
Hanqing Chao,
Amit Dhurandhar,
Pin-Yu Chen,
Ali Tajer,
Yangyang Xu,
Pingkun Yan
Abstract:
Domain shift is a common problem in clinical applications, where the training images (source domain) and the test images (target domain) are under different distributions. Unsupervised Domain Adaptation (UDA) techniques have been proposed to adapt models trained in the source domain to the target domain. However, those methods require a large number of images from the target domain for model train…
▽ More
Domain shift is a common problem in clinical applications, where the training images (source domain) and the test images (target domain) are under different distributions. Unsupervised Domain Adaptation (UDA) techniques have been proposed to adapt models trained in the source domain to the target domain. However, those methods require a large number of images from the target domain for model training. In this paper, we propose a novel method for Few-Shot Unsupervised Domain Adaptation (FSUDA), where only a limited number of unlabeled target domain samples are available for training. To accomplish this challenging task, first, a spectral sensitivity map is introduced to characterize the generalization weaknesses of models in the frequency domain. We then developed a Sensitivity-guided Spectral Adversarial MixUp (SAMix) method to generate target-style images to effectively suppresses the model sensitivity, which leads to improved model generalizability in the target domain. We demonstrated the proposed method and rigorously evaluated its performance on multiple tasks using several public datasets.
△ Less
Submitted 3 September, 2023;
originally announced September 2023.
-
Towards Carbon-Free Electricity: A Flow-Based Framework for Power Grid Carbon Accounting and Decarbonization
Authors:
Xin Chen,
Hungpo Chao,
Wenbo Shi,
Na Li
Abstract:
This paper introduces a comprehensive framework aimed at advancing research and policy development in the realm of decarbonization within electric power systems. The framework focuses on three key aspects: carbon accounting, carbon-aware decision-making, and carbon-electricity market design. It addresses existing problems, methods, and proposes solutions. In contrast to traditional pool-based emis…
▽ More
This paper introduces a comprehensive framework aimed at advancing research and policy development in the realm of decarbonization within electric power systems. The framework focuses on three key aspects: carbon accounting, carbon-aware decision-making, and carbon-electricity market design. It addresses existing problems, methods, and proposes solutions. In contrast to traditional pool-based emissions models, our framework proposes a novel flow-based emissions model. This model incorporates the underlying physical power grid and power flows, allowing for accurate carbon accounting at both temporal and spatial scales. This, in turn, facilitates informed decision-making to achieve grid decarbonization goals. The framework is built on a flow-based accounting methodology and utilizes the carbon-aware optimal power flow (C-OPF) technique as a theoretical foundation for decarbonization decision-making. Additionally, the paper explores the potential design of carbon-electricity markets and pricing mechanisms to incentivize decentralized decarbonization actions. The critical issues of data availability, infrastructure development, and considerations of fairness and equity are also discussed. This paper seeks to advance scholarly understanding and foster progress toward achieving sustainable and carbon-free electric power systems.
△ Less
Submitted 28 November, 2023; v1 submitted 6 August, 2023;
originally announced August 2023.
-
Virtual Screening of Chemical Space based on Quantum Annealing
Authors:
Takuro Tanaka,
Masami Sako,
Mahito Chiba,
Chul Lee,
Hyukgeun Cha,
Masayuki Ohzeki
Abstract:
For searching a new chemical material which satisfies the target characteristic value, for example emission wavelength, many cut and trial of experiments/calculations are required since the chemical space is astronomically large (organic molecules generates >10^60 candidates). Extracting feature importance is a method to reduce the chemical space, and limiting the search space to those features le…
▽ More
For searching a new chemical material which satisfies the target characteristic value, for example emission wavelength, many cut and trial of experiments/calculations are required since the chemical space is astronomically large (organic molecules generates >10^60 candidates). Extracting feature importance is a method to reduce the chemical space, and limiting the search space to those features leads to shorter development time. Quantum computer can generate sampling data faster than classical computers, and this property is utilized to extract feature importance. In this paper, quantum annealer was used as a sampler to make data for extracting feature importance of material properties. By screening the chemical space with feature importance, it was found that the chemical space can be reduced to less than 1 percent. This result suggests that the acceleration of material research can be achievable.
△ Less
Submitted 12 July, 2023;
originally announced July 2023.
-
Intruder configurations in $^{29}$Ne at the transition into the island of inversion: Detailed structure study of $^{28}$Ne
Authors:
H. Wang,
M. Yasuda,
Y. Kondo,
T. Nakamura,
J. A. Tostevin,
K. Ogata,
T. Otsuka,
A. Poves,
N. Shimizu,
K. Yoshida,
N. L. Achouri,
H. Al Falou,
L. Atar,
T. Aumann,
H. Baba,
K. Boretzky,
C. Caesar,
D. Calvet,
H. Chae,
N. Chiga,
A. Corsi,
H. L. Crawford,
F. Delaunay,
A. Delbart,
Q. Deshayes
, et al. (71 additional authors not shown)
Abstract:
Detailed $γ$-ray spectroscopy of the exotic neon isotope $^{28}$Ne has been performed for the first time using the one-neutron removal reaction from $^{29}$Ne on a liquid hydrogen target at 240~MeV/nucleon. Based on an analysis of parallel momentum distributions, a level scheme with spin-parity assignments has been constructed for $^{28}$Ne and the negative-parity states are identified for the fir…
▽ More
Detailed $γ$-ray spectroscopy of the exotic neon isotope $^{28}$Ne has been performed for the first time using the one-neutron removal reaction from $^{29}$Ne on a liquid hydrogen target at 240~MeV/nucleon. Based on an analysis of parallel momentum distributions, a level scheme with spin-parity assignments has been constructed for $^{28}$Ne and the negative-parity states are identified for the first time. The measured partial cross sections and momentum distributions reveal a significant intruder $p$-wave strength providing evidence of the breakdown of the $N=20$ and $N=28$ shell gaps. Only a weak, possible $f$-wave strength was observed to bound final states. Large-scale shell-model calculations with different effective interactions do not reproduce the large $p$-wave and small $f$-wave strength observed experimentally, indicating an ongoing challenge for a complete theoretical description of the transition into the island of inversion along the Ne isotopic chain.
△ Less
Submitted 28 June, 2023;
originally announced June 2023.
-
Rate-Splitting Multiple Access for 6G Networks: Ten Promising Scenarios and Applications
Authors:
Jeonghun Park,
Byungju Lee,
Jinseok Choi,
Hoon Lee,
Namyoon Lee,
Seok-Hwan Park,
Kyoung-Jae Lee,
Junil Choi,
Sung Ho Chae,
Sang-Woon Jeon,
Kyung Sup Kwak,
Bruno Clerckx,
Wonjae Shin
Abstract:
In the upcoming 6G era, multiple access (MA) will play an essential role in achieving high throughput performances required in a wide range of wireless applications. Since MA and interference management are closely related issues, the conventional MA techniques are limited in that they cannot provide near-optimal performance in universal interference regimes. Recently, rate-splitting multiple acce…
▽ More
In the upcoming 6G era, multiple access (MA) will play an essential role in achieving high throughput performances required in a wide range of wireless applications. Since MA and interference management are closely related issues, the conventional MA techniques are limited in that they cannot provide near-optimal performance in universal interference regimes. Recently, rate-splitting multiple access (RSMA) has been gaining much attention. RSMA splits an individual message into two parts: a common part, decodable by every user, and a private part, decodable only by the intended user. Each user first decodes the common message and then decodes its private message by applying successive interference cancellation (SIC). By doing so, RSMA not only embraces the existing MA techniques as special cases but also provides significant performance gains by efficiently mitigating inter-user interference in a broad range of interference regimes. In this article, we first present the theoretical foundation of RSMA. Subsequently, we put forth four key benefits of RSMA: spectral efficiency, robustness, scalability, and flexibility. Upon this, we describe how RSMA can enable ten promising scenarios and applications along with future research directions to pave the way for 6G.
△ Less
Submitted 22 June, 2023;
originally announced June 2023.
-
Learning Profitable NFT Image Diffusions via Multiple Visual-Policy Guided Reinforcement Learning
Authors:
Huiguo He,
Tianfu Wang,
Huan Yang,
Jianlong Fu,
Nicholas Jing Yuan,
Jian Yin,
Hongyang Chao,
Qi Zhang
Abstract:
We study the task of generating profitable Non-Fungible Token (NFT) images from user-input texts. Recent advances in diffusion models have shown great potential for image generation. However, existing works can fall short in generating visually-pleasing and highly-profitable NFT images, mainly due to the lack of 1) plentiful and fine-grained visual attribute prompts for an NFT image, and 2) effect…
▽ More
We study the task of generating profitable Non-Fungible Token (NFT) images from user-input texts. Recent advances in diffusion models have shown great potential for image generation. However, existing works can fall short in generating visually-pleasing and highly-profitable NFT images, mainly due to the lack of 1) plentiful and fine-grained visual attribute prompts for an NFT image, and 2) effective optimization metrics for generating high-quality NFT images. To solve these challenges, we propose a Diffusion-based generation framework with Multiple Visual-Policies as rewards (i.e., Diffusion-MVP) for NFT images. The proposed framework consists of a large language model (LLM), a diffusion-based image generator, and a series of visual rewards by design. First, the LLM enhances a basic human input (such as "panda") by generating more comprehensive NFT-style prompts that include specific visual attributes, such as "panda with Ninja style and green background." Second, the diffusion-based image generator is fine-tuned using a large-scale NFT dataset to capture fine-grained image styles and accessory compositions of popular NFT elements. Third, we further propose to utilize multiple visual-policies as optimization goals, including visual rarity levels, visual aesthetic scores, and CLIP-based text-image relevances. This design ensures that our proposed Diffusion-MVP is capable of minting NFT images with high visual quality and market value. To facilitate this research, we have collected the largest publicly available NFT image dataset to date, consisting of 1.5 million high-quality images with corresponding texts and market values. Extensive experiments including objective evaluations and user studies demonstrate that our framework can generate NFT images showing more visually engaging elements and higher market value, compared with SOTA approaches.
△ Less
Submitted 17 August, 2023; v1 submitted 20 June, 2023;
originally announced June 2023.
-
Single-particle-exact density functional theory
Authors:
Martin-Isbjörn Trappe,
Jun Hao Hue,
Jonah Huang Zi Chao,
Mikołaj Paraniak,
Djamila Hiller,
Jerzy Ciosłowski,
Berthold-Georg Englert
Abstract:
We introduce 'single-particle-exact density functional theory' (1pEx-DFT), a novel density functional approach that represents all single-particle contributions to the energy with exact functionals. Here, we parameterize interaction energy functionals by utilizing two new schemes for constructing density matrices from 'participation numbers' of the single-particle states of quantum many-body syste…
▽ More
We introduce 'single-particle-exact density functional theory' (1pEx-DFT), a novel density functional approach that represents all single-particle contributions to the energy with exact functionals. Here, we parameterize interaction energy functionals by utilizing two new schemes for constructing density matrices from 'participation numbers' of the single-particle states of quantum many-body systems. These participation numbers play the role of the variational variables akin to the particle densities in standard orbital-free density functional theory. We minimize the total energies with the help of evolutionary algorithms and obtain ground-state energies that are typically accurate at the one-percent level for our proof-of-principle simulations that comprise interacting Fermi gases as well as the electronic structure of atoms and ions, with and without relativistic corrections. We thereby illustrate the ingredients and practical features of 1pEx-DFT and reveal its potential of becoming an accurate, scalable, and transferable technology for simulating mesoscopic quantum many-body systems.
△ Less
Submitted 16 October, 2023; v1 submitted 4 May, 2023;
originally announced May 2023.
-
Reconfigurable Intelligent Surface Aided Hybrid Beamforming: Optimal Placement and Beamforming Design
Authors:
Najam Us Saqib,
Shumei Hou,
Sung Ho Chae,
Sang-Woon Jeon
Abstract:
We consider reconfigurable intelligent surface (RIS) aided sixth-generation (6G) terahertz (THz) communications for indoor environment in which a base station (BS) wishes to send independent messages to its serving users with the help of multiple RISs. For indoor environment, various obstacles such as pillars, walls, and other objects can result in no line-of-sight signal path between the BS and a…
▽ More
We consider reconfigurable intelligent surface (RIS) aided sixth-generation (6G) terahertz (THz) communications for indoor environment in which a base station (BS) wishes to send independent messages to its serving users with the help of multiple RISs. For indoor environment, various obstacles such as pillars, walls, and other objects can result in no line-of-sight signal path between the BS and a user, which can significantly degrade performance. To overcome such limitation of indoor THz communication, we firstly optimize the placement of RISs to maximize the coverage area. Under the optimized RIS placement, we propose 3D hybrid beamforming at the BS and phase adjustment at RISs, which are jointly performed at the BS and RISs via codebook-based 3D beam scanning with low complexity. Numerical simulations demonstrate that the proposed scheme significantly improves the average sum rate compared to the cases of no RIS and randomly deployed RISs. It is further shown that the proposed codebook-based 3D beam scanning efficiently aligns analog beams between BS--user links or BS--RIS--user links and, as a consequence, achieves the average sum rate close to that of coherent beam alignment requiring global channel state information.
△ Less
Submitted 21 March, 2023;
originally announced March 2023.
-
Residual Physics Learning and System Identification for Sim-to-real Transfer of Policies on Buoyancy Assisted Legged Robots
Authors:
Nitish Sontakke,
Hosik Chae,
Sangjoon Lee,
Tianle Huang,
Dennis W. Hong,
Sehoon Ha
Abstract:
The light and soft characteristics of Buoyancy Assisted Lightweight Legged Unit (BALLU) robots have a great potential to provide intrinsically safe interactions in environments involving humans, unlike many heavy and rigid robots. However, their unique and sensitive dynamics impose challenges to obtaining robust control policies in the real world. In this work, we demonstrate robust sim-to-real tr…
▽ More
The light and soft characteristics of Buoyancy Assisted Lightweight Legged Unit (BALLU) robots have a great potential to provide intrinsically safe interactions in environments involving humans, unlike many heavy and rigid robots. However, their unique and sensitive dynamics impose challenges to obtaining robust control policies in the real world. In this work, we demonstrate robust sim-to-real transfer of control policies on the BALLU robots via system identification and our novel residual physics learning method, Environment Mimic (EnvMimic). First, we model the nonlinear dynamics of the actuators by collecting hardware data and optimizing the simulation parameters. Rather than relying on standard supervised learning formulations, we utilize deep reinforcement learning to train an external force policy to match real-world trajectories, which enables us to model residual physics with greater fidelity. We analyze the improved simulation fidelity by comparing the simulation trajectories against the real-world ones. We finally demonstrate that the improved simulator allows us to learn better walking and turning policies that can be successfully deployed on the hardware of BALLU.
△ Less
Submitted 16 March, 2023;
originally announced March 2023.
-
Infrared Ejecta and Cold Dust in the Young Supernova Remnant N132D
Authors:
Jeonghee Rho,
Aravind P. Ravi,
Jonathan D. Slavin,
Heechan Cha
Abstract:
We present Spitzer, WISE, and Herschel observations of the young supernova remnant (SNR) N132D in the LMC, including 3-40 microns Spitzer IRS mapping, 12 microns WISE and 70, 100, 160, 250, 350, and 500 microns Herschel images. The high-velocity lines of [Ne II] at 12.8 microns, [Ne III] at 15.5 microns, and [O IV] 26 microns reveal infrared ejecta concentrated in a central ring and coincide the o…
▽ More
We present Spitzer, WISE, and Herschel observations of the young supernova remnant (SNR) N132D in the LMC, including 3-40 microns Spitzer IRS mapping, 12 microns WISE and 70, 100, 160, 250, 350, and 500 microns Herschel images. The high-velocity lines of [Ne II] at 12.8 microns, [Ne III] at 15.5 microns, and [O IV] 26 microns reveal infrared ejecta concentrated in a central ring and coincide the optical and X-ray ejecta. Herschel images reveal far-IR emission coinciding with the central ejecta, which suggests that the IR emission is freshly formed, cold dust in the SN-ejecta. The infrared spectra are remarkably similar to those of another young SNR of 1E0102 with Ne and O lines. Shock modeling of the Ne ejecta emission suggests a gas temperature of 300 - 600 K and densities in the range 1000-20,000 cm^{-3} in the post-shock photoionized region. The IR continuum from the ejecta shows an 18 microns-peak dust feature. We performed spectral fitting to the IRS dust continuum and Herschel photometry. The dust mass associated with the central ejecta is 1.25+-0.65 Msun, while the 18 microns dust feature requires forsterite grains. The dust mass of the central ejecta region in N132D is higher than those of other young SNRs, which is likely associated with its higher progenitor mass. We discuss the dust productivity in the ejecta of N132D and infer its plausible implications for the dust in the early Universe.
△ Less
Submitted 8 March, 2023;
originally announced March 2023.
-
InFusionSurf: Refining Neural RGB-D Surface Reconstruction Using Per-Frame Intrinsic Refinement and TSDF Fusion Prior Learning
Authors:
Seunghwan Lee,
Gwanmo Park,
Hyewon Son,
Jiwon Ryu,
Han Joo Chae
Abstract:
We introduce InFusionSurf, a novel approach to enhance the fidelity of neural radiance field (NeRF) frameworks for 3D surface reconstruction using RGB-D video frames. Building upon previous methods that have employed feature encoding to improve optimization speed, we further improve the reconstruction quality with minimal impact on optimization time by refining depth information. Our per-frame int…
▽ More
We introduce InFusionSurf, a novel approach to enhance the fidelity of neural radiance field (NeRF) frameworks for 3D surface reconstruction using RGB-D video frames. Building upon previous methods that have employed feature encoding to improve optimization speed, we further improve the reconstruction quality with minimal impact on optimization time by refining depth information. Our per-frame intrinsic refinement scheme addresses frame-specific blurs caused by camera motion in each depth frame. Furthermore, InFusionSurf utilizes a classical real-time 3D surface reconstruction method, the truncated signed distance field (TSDF) Fusion, as prior knowledge to pretrain the feature grid to support reconstruction details while accelerating the training. The quantitative and qualitative experiments comparing the performances of InFusionSurf against prior work indicate that our method is capable of accurately reconstructing a scene without sacrificing optimization speed. We also demonstrate the effectiveness of our per-frame intrinsic refinement and TSDF Fusion prior learning techniques via an ablation study.
△ Less
Submitted 3 September, 2023; v1 submitted 8 March, 2023;
originally announced March 2023.
-
CoTEVer: Chain of Thought Prompting Annotation Toolkit for Explanation Verification
Authors:
Seungone Kim,
Se June Joo,
Yul Jang,
Hyungjoo Chae,
Jinyoung Yeo
Abstract:
Chain-of-thought (CoT) prompting enables large language models (LLMs) to solve complex reasoning tasks by generating an explanation before the final prediction. Despite it's promising ability, a critical downside of CoT prompting is that the performance is greatly affected by the factuality of the generated explanation. To improve the correctness of the explanations, fine-tuning language models wi…
▽ More
Chain-of-thought (CoT) prompting enables large language models (LLMs) to solve complex reasoning tasks by generating an explanation before the final prediction. Despite it's promising ability, a critical downside of CoT prompting is that the performance is greatly affected by the factuality of the generated explanation. To improve the correctness of the explanations, fine-tuning language models with explanation data is needed. However, there exists only a few datasets that can be used for such approaches, and no data collection tool for building them. Thus, we introduce CoTEVer, a tool-kit for annotating the factual correctness of generated explanations and collecting revision data of wrong explanations. Furthermore, we suggest several use cases where the data collected with CoTEVer can be utilized for enhancing the faithfulness of explanations. Our toolkit is publicly available at https://github.com/SeungoneKim/CoTEVer.
△ Less
Submitted 6 March, 2023;
originally announced March 2023.
-
TUTORING: Instruction-Grounded Conversational Agent for Language Learners
Authors:
Hyungjoo Chae,
Minjin Kim,
Chaehyeong Kim,
Wonseok Jeong,
Hyejoong Kim,
Junmyung Lee,
Jinyoung Yeo
Abstract:
In this paper, we propose Tutoring bot, a generative chatbot trained on a large scale of tutor-student conversations for English-language learning. To mimic a human tutor's behavior in language education, the tutor bot leverages diverse educational instructions and grounds to each instruction as additional input context for the tutor response generation. As a single instruction generally involves…
▽ More
In this paper, we propose Tutoring bot, a generative chatbot trained on a large scale of tutor-student conversations for English-language learning. To mimic a human tutor's behavior in language education, the tutor bot leverages diverse educational instructions and grounds to each instruction as additional input context for the tutor response generation. As a single instruction generally involves multiple dialogue turns to give the student sufficient speaking practice, the tutor bot is required to monitor and capture when the current instruction should be kept or switched to the next instruction. For that, the tutor bot is learned to not only generate responses but also infer its teaching action and progress on the current conversation simultaneously by a multi-task learning scheme. Our Tutoring bot is deployed under a non-commercial use license at https://tutoringai.com.
△ Less
Submitted 24 February, 2023;
originally announced February 2023.
-
Harmonic (Quantum) Neural Networks
Authors:
Atiyo Ghosh,
Antonio A. Gentile,
Mario Dagrada,
Chul Lee,
Seong-Hyok Kim,
Hyukgeun Cha,
Yunjun Choi,
Brad Kim,
Jeong-Il Kye,
Vincent E. Elfving
Abstract:
Harmonic functions are abundant in nature, appearing in limiting cases of Maxwell's, Navier-Stokes equations, the heat and the wave equation. Consequently, there are many applications of harmonic functions from industrial process optimisation to robotic path planning and the calculation of first exit times of random walks. Despite their ubiquity and relevance, there have been few attempts to incor…
▽ More
Harmonic functions are abundant in nature, appearing in limiting cases of Maxwell's, Navier-Stokes equations, the heat and the wave equation. Consequently, there are many applications of harmonic functions from industrial process optimisation to robotic path planning and the calculation of first exit times of random walks. Despite their ubiquity and relevance, there have been few attempts to incorporate inductive biases towards harmonic functions in machine learning contexts. In this work, we demonstrate effective means of representing harmonic functions in neural networks and extend such results also to quantum neural networks to demonstrate the generality of our approach. We benchmark our approaches against (quantum) physics-informed neural networks, where we show favourable performance.
△ Less
Submitted 13 August, 2023; v1 submitted 14 December, 2022;
originally announced December 2022.
-
Semantic-Conditional Diffusion Networks for Image Captioning
Authors:
Jianjie Luo,
Yehao Li,
Yingwei Pan,
Ting Yao,
Jianlin Feng,
Hongyang Chao,
Tao Mei
Abstract:
Recent advances on text-to-image generation have witnessed the rise of diffusion models which act as powerful generative models. Nevertheless, it is not trivial to exploit such latent variable models to capture the dependency among discrete words and meanwhile pursue complex visual-language alignment in image captioning. In this paper, we break the deeply rooted conventions in learning Transformer…
▽ More
Recent advances on text-to-image generation have witnessed the rise of diffusion models which act as powerful generative models. Nevertheless, it is not trivial to exploit such latent variable models to capture the dependency among discrete words and meanwhile pursue complex visual-language alignment in image captioning. In this paper, we break the deeply rooted conventions in learning Transformer-based encoder-decoder, and propose a new diffusion model based paradigm tailored for image captioning, namely Semantic-Conditional Diffusion Networks (SCD-Net). Technically, for each input image, we first search the semantically relevant sentences via cross-modal retrieval model to convey the comprehensive semantic information. The rich semantics are further regarded as semantic prior to trigger the learning of Diffusion Transformer, which produces the output sentence in a diffusion process. In SCD-Net, multiple Diffusion Transformer structures are stacked to progressively strengthen the output sentence with better visional-language alignment and linguistical coherence in a cascaded manner. Furthermore, to stabilize the diffusion process, a new self-critical sequence training strategy is designed to guide the learning of SCD-Net with the knowledge of a standard autoregressive Transformer model. Extensive experiments on COCO dataset demonstrate the promising potential of using diffusion models in the challenging image captioning task. Source code is available at \url{https://github.com/YehLi/xmodaler/tree/master/configs/image_caption/scdnet}.
△ Less
Submitted 6 December, 2022;
originally announced December 2022.
-
When Neural Networks Fail to Generalize? A Model Sensitivity Perspective
Authors:
Jiajin Zhang,
Hanqing Chao,
Amit Dhurandhar,
Pin-Yu Chen,
Ali Tajer,
Yangyang Xu,
Pingkun Yan
Abstract:
Domain generalization (DG) aims to train a model to perform well in unseen domains under different distributions. This paper considers a more realistic yet more challenging scenario,namely Single Domain Generalization (Single-DG), where only a single source domain is available for training. To tackle this challenge, we first try to understand when neural networks fail to generalize? We empirically…
▽ More
Domain generalization (DG) aims to train a model to perform well in unseen domains under different distributions. This paper considers a more realistic yet more challenging scenario,namely Single Domain Generalization (Single-DG), where only a single source domain is available for training. To tackle this challenge, we first try to understand when neural networks fail to generalize? We empirically ascertain a property of a model that correlates strongly with its generalization that we coin as "model sensitivity". Based on our analysis, we propose a novel strategy of Spectral Adversarial Data Augmentation (SADA) to generate augmented images targeted at the highly sensitive frequencies. Models trained with these hard-to-learn samples can effectively suppress the sensitivity in the frequency space, which leads to improved generalization performance. Extensive experiments on multiple public datasets demonstrate the superiority of our approach, which surpasses the state-of-the-art single-DG methods.
△ Less
Submitted 1 December, 2022;
originally announced December 2022.
-
Internet of Behaviors: A Survey
Authors:
Jiayi Sun,
Wensheng Gan,
Han-Chieh Chao,
Philip S. Yu,
Weiping Ding
Abstract:
The Internet of Behavior is a research theme that aims to analyze human behavior data on the Internet from the perspective of behavioral psychology, obtain insights about human behavior, and better understand the intention behind the behavior. In this way, the Internet of Behavior can predict human behavioral trends in the future and even change human behavior, which can provide more convenience f…
▽ More
The Internet of Behavior is a research theme that aims to analyze human behavior data on the Internet from the perspective of behavioral psychology, obtain insights about human behavior, and better understand the intention behind the behavior. In this way, the Internet of Behavior can predict human behavioral trends in the future and even change human behavior, which can provide more convenience for human life. With the increasing prosperity of the Internet of Things, more and more behavior-related data is collected on the Internet by connected devices such as sensors. People and behavior are connected through the extension of the Internet of Things -- the Internet of Behavior. At present, the Internet of Behavior has gradually been applied to our lives, but it is still in its early stages, and many opportunities and challenges are emerging. This paper provides an in-depth overview of the fundamental aspects of the Internet of Behavior: (1) We introduce the development process and research status of the Internet of Behavior from the perspective of the Internet of Things. (2) We propose the characteristics of the Internet of Behavior and define its development direction in terms of three aspects: real-time, autonomy, and reliability. (3) We provide a comprehensive summary of the current applications of the Internet of Behavior, including specific discussions in five scenarios that give an overview of the application status of the Internet of Behavior. (4) We discuss the challenges of the Internet of Behavior's development and its future directions, which hopefully will bring some progress to the Internet of Behavior. To the best of our knowledge, this is the first survey paper on the Internet of Behavior. We hope that this in-depth review can provide some useful directions for more productive research in related fields.
△ Less
Submitted 28 November, 2022;
originally announced November 2022.
-
Metaverse in Education: Vision, Opportunities, and Challenges
Authors:
Hong Lin,
Shicheng Wan,
Wensheng Gan,
Jiahui Chen,
Han-Chieh Chao
Abstract:
Traditional education has been updated with the development of information technology in human history. Within big data and cyber-physical systems, the Metaverse has generated strong interest in various applications (e.g., entertainment, business, and cultural travel) over the last decade. As a novel social work idea, the Metaverse consists of many kinds of technologies, e.g., big data, interactio…
▽ More
Traditional education has been updated with the development of information technology in human history. Within big data and cyber-physical systems, the Metaverse has generated strong interest in various applications (e.g., entertainment, business, and cultural travel) over the last decade. As a novel social work idea, the Metaverse consists of many kinds of technologies, e.g., big data, interaction, artificial intelligence, game design, Internet computing, Internet of Things, and blockchain. It is foreseeable that the usage of Metaverse will contribute to educational development. However, the architectures of the Metaverse in education are not yet mature enough. There are many questions we should address for the Metaverse in education. To this end, this paper aims to provide a systematic literature review of Metaverse in education. This paper is a comprehensive survey of the Metaverse in education, with a focus on current technologies, challenges, opportunities, and future directions. First, we present a brief overview of the Metaverse in education, as well as the motivation behind its integration. Then, we survey some important characteristics for the Metaverse in education, including the personal teaching environment and the personal learning environment. Next, we envisage what variations of this combination will bring to education in the future and discuss their strengths and weaknesses. We also review the state-of-the-art case studies (including technical companies and educational institutions) for Metaverse in education. Finally, we point out several challenges and issues in this promising area.
△ Less
Submitted 27 November, 2022;
originally announced November 2022.
-
Not All Asians are the Same: A Disaggregated Approach to Identifying Anti-Asian Racism in Social Media
Authors:
Fan Wu,
Sanyam Lakhanpal,
Qian Li,
Kookjin Lee,
Doowon Kim,
Heewon Chae,
Hazel K. Kwon
Abstract:
Recent policy initiatives have acknowledged the importance of disaggregating data pertaining to diverse Asian ethnic communities to gain a more comprehensive understanding of their current status and to improve their overall well-being. However, research on anti-Asian racism has thus far fallen short of properly incorporating data disaggregation practices. Our study addresses this gap by collectin…
▽ More
Recent policy initiatives have acknowledged the importance of disaggregating data pertaining to diverse Asian ethnic communities to gain a more comprehensive understanding of their current status and to improve their overall well-being. However, research on anti-Asian racism has thus far fallen short of properly incorporating data disaggregation practices. Our study addresses this gap by collecting 12-month-long data from X (formerly known as Twitter) that contain diverse sub-ethnic group representations within Asian communities. In this dataset, we break down anti-Asian toxic messages based on both temporal and ethnic factors and conduct a series of comparative analyses of toxic messages, targeting different ethnic groups. Using temporal persistence analysis, $n$-gram-based correspondence analysis, and topic modeling, this study provides compelling evidence that anti-Asian messages comprise various distinctive narratives. Certain messages targeting sub-ethnic Asian groups entail different topics that distinguish them from those targeting Asians in a generic manner or those aimed at major ethnic groups, such as Chinese and Indian. By introducing several techniques that facilitate comparisons of online anti-Asian hate towards diverse ethnic communities, this study highlights the importance of taking a nuanced and disaggregated approach for understanding racial hatred to formulate effective mitigation strategies.
△ Less
Submitted 12 February, 2024; v1 submitted 20 October, 2022;
originally announced October 2022.
-
Fleet-Level Environmental Assessments for Feasibility of Aviation Emission Reduction Goals
Authors:
Kolawole Ogunsina,
Hsun Chao,
Nithin Jojo Kolencherry,
Samarth Jain,
Kushal Moolchandani,
Daniel DeLaurentis,
William Crossley
Abstract:
The International Air Transport Association (IATA) is one of several organizations that have presented goals for future CO2 emissions from commercial aviation with the intent of alleviating the associated environmental impacts. These goals include attaining carbon-neutral growth in the year 2020 and total aviation CO2 emissions in 2050 equal to 50% of 2005 aviation CO2 emissions. This paper presen…
▽ More
The International Air Transport Association (IATA) is one of several organizations that have presented goals for future CO2 emissions from commercial aviation with the intent of alleviating the associated environmental impacts. These goals include attaining carbon-neutral growth in the year 2020 and total aviation CO2 emissions in 2050 equal to 50% of 2005 aviation CO2 emissions. This paper presents the use of a simulation-based approach to predict future CO2 emissions from commercial aviation based upon a set of scenarios developed as part of the Aircraft Technology Modeling and Assessment project within ASCENT, the FAA Center of Excellence for Alternative Jet Fuels and the Environment. Results indicate that, in future scenarios with increasing demand for air travel, it is difficult to reduce CO2 emissions in 2050 to levels equal to or below 2005 levels, although neutral CO2 growth after 2020 may be possible.
△ Less
Submitted 16 September, 2022;
originally announced October 2022.
-
Metaverse: Survey, Applications, Security, and Opportunities
Authors:
Jiayi Sun,
Wensheng Gan,
Han-Chieh Chao,
Philip S. Yu
Abstract:
As a fusion of various emerging digital technologies, the Metaverse aims to build a virtual shared digital space. It is closely related to extended reality, digital twin, blockchain, and other technologies. Its goal is to build a digital space based on the real world, form a virtual economic system, and expand the space of human activities, which injects new vitality into the social, economic, and…
▽ More
As a fusion of various emerging digital technologies, the Metaverse aims to build a virtual shared digital space. It is closely related to extended reality, digital twin, blockchain, and other technologies. Its goal is to build a digital space based on the real world, form a virtual economic system, and expand the space of human activities, which injects new vitality into the social, economic, and other fields. In this article, we make the following contributions. We first introduce the basic concepts such as the development process, definition, and characteristics of the Metaverse. After that, we analyze the overall framework and supporting technologies of the Metaverse and summarize the status of common fields. Finally, we focus on the security and privacy issues that exist in the Metaverse, and give corresponding solutions or directions. We also discuss the challenges and possible research directions of the Metaverse. We believe this comprehensive review can provide an overview of the Metaverse and research directions for some multidisciplinary studies.
△ Less
Submitted 14 October, 2022;
originally announced October 2022.
-
Out-of-Distribution Detection with Hilbert-Schmidt Independence Optimization
Authors:
Jingyang Lin,
Yu Wang,
Qi Cai,
Yingwei Pan,
Ting Yao,
Hongyang Chao,
Tao Mei
Abstract:
Outlier detection tasks have been playing a critical role in AI safety. There has been a great challenge to deal with this task. Observations show that deep neural network classifiers usually tend to incorrectly classify out-of-distribution (OOD) inputs into in-distribution classes with high confidence. Existing works attempt to solve the problem by explicitly imposing uncertainty on classifiers w…
▽ More
Outlier detection tasks have been playing a critical role in AI safety. There has been a great challenge to deal with this task. Observations show that deep neural network classifiers usually tend to incorrectly classify out-of-distribution (OOD) inputs into in-distribution classes with high confidence. Existing works attempt to solve the problem by explicitly imposing uncertainty on classifiers when OOD inputs are exposed to the classifier during training. In this paper, we propose an alternative probabilistic paradigm that is both practically useful and theoretically viable for the OOD detection tasks. Particularly, we impose statistical independence between inlier and outlier data during training, in order to ensure that inlier data reveals little information about OOD data to the deep estimator during training. Specifically, we estimate the statistical dependence between inlier and outlier data through the Hilbert-Schmidt Independence Criterion (HSIC), and we penalize such metric during training. We also associate our approach with a novel statistical test during the inference time coupled with our principled motivation. Empirical results show that our method is effective and robust for OOD detection on various benchmarks. In comparison to SOTA models, our approach achieves significant improvement regarding FPR95, AUROC, and AUPR metrics. Code is available: \url{https://github.com/jylins/hood}.
△ Less
Submitted 26 September, 2022;
originally announced September 2022.
-
2D material platform for overcoming the amplitude-phase tradeoff in ring modulators
Authors:
Ipshita Datta,
Andres Gil-Molina,
Sang Hoon Chae,
James Hone,
Michal Lipson
Abstract:
Compact, high-speed electro-optic phase modulators play a vital role in various large-scale applications including phased arrays, quantum and neural networks, and optical communication links. Conventional phase modulators suffer from a fundamental tradeoff between device length and optical loss that limits their scaling capabilities. High-finesse ring resonators have been traditionally used as com…
▽ More
Compact, high-speed electro-optic phase modulators play a vital role in various large-scale applications including phased arrays, quantum and neural networks, and optical communication links. Conventional phase modulators suffer from a fundamental tradeoff between device length and optical loss that limits their scaling capabilities. High-finesse ring resonators have been traditionally used as compact intensity modulators, but their use for phase modulation have been limited due to the high insertion loss associated with the phase change. Here, we show that high-finesse resonators can achieve a strong phase change with low insertion loss by simultaneous modulation of the real and imaginary parts of the refractive index, to the same extent i.e. $\frac{Δn}{Δk} \sim 1$. To implement this strategy, we utilize a hybrid platform that combines a low-loss SiN ring resonator with electro-absorptive graphene (Gr) and electro-refractive WSe$_2$. We achieve a phase modulation efficiency ($V_{\fracπ{2}} \cdot L_{\fracπ{2}}$) of 0.045 V $\cdot$ cm with an insertion loss (IL$_{\fracπ{2}}$) of 4.7 dB for a phase change of $\fracπ{2}$ radians, in a 25 $μ$m long Gr-Al$_2$O$_3$-WSe$_2$ capacitor embedded on a SiN ring of 50 $μ$m radius. We find that our Gr-Al$_2$O$_3$-WSe$_2$ capacitor can support an electro-optic bandwidth of 14.9 $\pm$ 0.1 GHz. We further show that the $V_{\fracπ{2}} \cdot L_{\fracπ{2}}$ of our SiN-2D platform is at least an order of magnitude lower than that of electro-optic phase modulators based on silicon, III-V on silicon, graphene on silicon and lithium niobate. This SiN-2D hybrid platform provides the impetus to design compact and high-speed reconfigurable circuits with graphene and transition metal dichalcogenide (TMD) monolayers that can enable large-scale photonic systems.
△ Less
Submitted 1 December, 2022; v1 submitted 17 September, 2022;
originally announced September 2022.
-
DS-K3DOM: 3-D Dynamic Occupancy Mapping with Kernel Inference and Dempster-Shafer Evidential Theory
Authors:
Juyeop Han,
Youngjae Min,
Hyeok-Joo Chae,
Byeong-Min Jeong,
Han-Lim Choi
Abstract:
Occupancy mapping has been widely utilized to represent the surroundings for autonomous robots to perform tasks such as navigation and manipulation. While occupancy mapping in 2-D environments has been well-studied, there have been few approaches suitable for 3-D dynamic occupancy mapping which is essential for aerial robots. This paper presents a novel 3-D dynamic occupancy mapping algorithm call…
▽ More
Occupancy mapping has been widely utilized to represent the surroundings for autonomous robots to perform tasks such as navigation and manipulation. While occupancy mapping in 2-D environments has been well-studied, there have been few approaches suitable for 3-D dynamic occupancy mapping which is essential for aerial robots. This paper presents a novel 3-D dynamic occupancy mapping algorithm called DS-K3DOM. We first establish a Bayesian method to sequentially update occupancy maps for a stream of measurements based on the random finite set theory. Then, we approximate it with particles in the Dempster-Shafer domain to enable real-time computation. Moreover, the algorithm applies kernel-based inference with Dirichlet basic belief assignment to enable dense mapping from sparse measurements. The efficacy of the proposed algorithm is demonstrated through simulations and real experiments.
△ Less
Submitted 24 February, 2023; v1 submitted 16 September, 2022;
originally announced September 2022.
-
Integrated Multiport Bidirectional DC-DC Converter for HEV/FCV Applications
Authors:
Bang Le-Huy Nguyen,
Honnyong Cha,
Tuyen Vu,
Thai-Thanh Nguyen
Abstract:
This paper proposes a novel integrated multiport bidirectional dc-dc converter to interface the battery, the ultra-capacitor, the fuel cell, or other energy sources with the dc-link capacitor of the hybrid energy systems such as the hybrid electric vehicle (HEV) and fuel cell vehicle (FCV) applications. The proposed converter can be applied to the distributed generation systems which include local…
▽ More
This paper proposes a novel integrated multiport bidirectional dc-dc converter to interface the battery, the ultra-capacitor, the fuel cell, or other energy sources with the dc-link capacitor of the hybrid energy systems such as the hybrid electric vehicle (HEV) and fuel cell vehicle (FCV) applications. The proposed converter can be applied to the distributed generation systems which include local energy sources, storage, and loads. It can perform both buck and boost functions with fewer switches. In addition, it is extendable when more inputs and/or outputs are required. The operating principle and control strategy of the proposed converter will be analyzed in detail. For verification, simulation, and experimental results of the four utilized operating modes of an HEV/FCV are provided.
△ Less
Submitted 13 September, 2022;
originally announced September 2022.
-
IMG2IMU: Translating Knowledge from Large-Scale Images to IMU Sensing Applications
Authors:
Hyungjun Yoon,
Hyeongheon Cha,
Hoang C. Nguyen,
Taesik Gong,
Sung-Ju Lee
Abstract:
Pre-training representations acquired via self-supervised learning could achieve high accuracy on even tasks with small training data. Unlike in vision and natural language processing domains, pre-training for IMU-based applications is challenging, as there are few public datasets with sufficient size and diversity to learn generalizable representations. To overcome this problem, we propose IMG2IM…
▽ More
Pre-training representations acquired via self-supervised learning could achieve high accuracy on even tasks with small training data. Unlike in vision and natural language processing domains, pre-training for IMU-based applications is challenging, as there are few public datasets with sufficient size and diversity to learn generalizable representations. To overcome this problem, we propose IMG2IMU that adapts pre-trained representation from large-scale images to diverse IMU sensing tasks. We convert the sensor data into visually interpretable spectrograms for the model to utilize the knowledge gained from vision. We further present a sensor-aware pre-training method for images that enables models to acquire particularly impactful knowledge for IMU sensing applications. This involves using contrastive learning on our augmentation set customized for the properties of sensor data. Our evaluation with four different IMU sensing tasks shows that IMG2IMU outperforms the baselines pre-trained on sensor data by an average of 9.6%p F1-score, illustrating that vision knowledge can be usefully incorporated into IMU sensing applications where only limited training data is available.
△ Less
Submitted 29 February, 2024; v1 submitted 2 September, 2022;
originally announced September 2022.
-
Mind the Gap! Injecting Commonsense Knowledge for Abstractive Dialogue Summarization
Authors:
Seungone Kim,
Se June Joo,
Hyungjoo Chae,
Chaehyeong Kim,
Seung-won Hwang,
Jinyoung Yeo
Abstract:
In this paper, we propose to leverage the unique characteristics of dialogues sharing commonsense knowledge across participants, to resolve the difficulties in summarizing them. We present SICK, a framework that uses commonsense inferences as additional context. Compared to previous work that solely relies on the input dialogue, SICK uses an external knowledge model to generate a rich set of commo…
▽ More
In this paper, we propose to leverage the unique characteristics of dialogues sharing commonsense knowledge across participants, to resolve the difficulties in summarizing them. We present SICK, a framework that uses commonsense inferences as additional context. Compared to previous work that solely relies on the input dialogue, SICK uses an external knowledge model to generate a rich set of commonsense inferences and selects the most probable one with a similarity-based selection method. Built upon SICK, SICK++ utilizes commonsense as supervision, where the task of generating commonsense inferences is added upon summarizing the dialogue in a multi-task learning setting. Experimental results show that with injected commonsense knowledge, our framework generates more informative and consistent summaries than existing methods.
△ Less
Submitted 2 September, 2022;
originally announced September 2022.
-
Toward cm-Level Accuracy: Carrier Phase Positioning for IIoT in 5G-Advanced NR Networks
Authors:
Abdurrahman Fouda,
Ryan Keating,
Hyun-Su Cha
Abstract:
High-precision positioning accuracy is among the key features of the future fifth-generation (5G-advanced) cellular networks to enable a wide variety of commercial, critical, and consumer use cases. 5G new radio (NR) systems have relied on (1) cellular temporal/angular-based positioning methods to provide the indoor environments with a moderate positioning accuracy that is well below the positioni…
▽ More
High-precision positioning accuracy is among the key features of the future fifth-generation (5G-advanced) cellular networks to enable a wide variety of commercial, critical, and consumer use cases. 5G new radio (NR) systems have relied on (1) cellular temporal/angular-based positioning methods to provide the indoor environments with a moderate positioning accuracy that is well below the positioning requirements of these future use cases and (2) highly precise satellite carrier phase/code-based positioning methods for the outdoor deployments that are limited by the availability of the satellite coverage. This paper defines the relevant standard mechanisms and algorithms to use the carrier phase cellular-based measurements as a potential solution to achieve a high-precision positioning estimation accuracy in 5G-advanced NR networks. The presented positioning technique is evaluated using high-fidelity system-level simulations for indoor factory (InF) deployment scenarios. The numerical results demonstrate that the presented technique can significantly improve the positioning accuracy compared with the state-of-the-art NR positioning methods. Our findings in this paper also show that the carrier phase method not only provides an indoor complement to the outdoor satellite positioning but also provides an outdoor alternative to the high-precision satellite methods.
△ Less
Submitted 13 July, 2022;
originally announced July 2022.
-
Regression Metric Loss: Learning a Semantic Representation Space for Medical Images
Authors:
Hanqing Chao,
Jiajin Zhang,
Pingkun Yan
Abstract:
Regression plays an essential role in many medical imaging applications for estimating various clinical risk or measurement scores. While training strategies and loss functions have been studied for the deep neural networks in medical image classification tasks, options for regression tasks are very limited. One of the key challenges is that the high-dimensional feature representation learned by e…
▽ More
Regression plays an essential role in many medical imaging applications for estimating various clinical risk or measurement scores. While training strategies and loss functions have been studied for the deep neural networks in medical image classification tasks, options for regression tasks are very limited. One of the key challenges is that the high-dimensional feature representation learned by existing popular loss functions like Mean Squared Error or L1 loss is hard to interpret. In this paper, we propose a novel Regression Metric Loss (RM-Loss), which endows the representation space with the semantic meaning of the label space by finding a representation manifold that is isometric to the label space. Experiments on two regression tasks, i.e. coronary artery calcium score estimation and bone age assessment, show that RM-Loss is superior to the existing popular regression losses on both performance and interpretability. Code is available at https://github.com/DIAL-RPI/Regression-Metric-Loss.
△ Less
Submitted 11 July, 2022;
originally announced July 2022.