Showing 101–150 of 772 results for author: He, Q

Nonlinear Regression Analysis

Authors: Hsin-Hsiung Huang, Qing He

Abstract: Nonlinear regression analysis is a popular and important tool for scientists and engineers. In this article, we introduce theories and methods of nonlinear regression and its statistical inferences using the frequentist and Bayesian statistical modeling and computation. Least squares with the Gauss-Newton method is the most widely used approach to parameters estimation. Under the assumption of nor… ▽ More Nonlinear regression analysis is a popular and important tool for scientists and engineers. In this article, we introduce theories and methods of nonlinear regression and its statistical inferences using the frequentist and Bayesian statistical modeling and computation. Least squares with the Gauss-Newton method is the most widely used approach to parameters estimation. Under the assumption of normally distributed errors, maximum likelihood estimation is equivalent to least squares estimation. The Wald confidence regions for parameters in a nonlinear regression model are affected by the curvatures in the mean function. Furthermore, we introduce the Newton-Raphson method and the generalized least squares method to deal with variance heterogeneity. Examples of simulation data analysis are provided to illustrate important properties of confidence regions and the statistical inferences using the nonlinear least squares estimation and Bayesian inference. △ Less

Submitted 7 February, 2024; originally announced February 2024.

A Framework of Zero-Inflated Bayesian Negative Binomial Regression Models For Spatiotemporal Data

Authors: Qing He, Hsin-Hsiung Huang

Abstract: Spatiotemporal data analysis with massive zeros is widely used in many areas such as epidemiology and public health. We use a Bayesian framework to fit zero-inflated negative binomial models and employ a set of latent variables from Pólya-Gamma distributions to derive an efficient Gibbs sampler. The proposed model accommodates varying spatial and temporal random effects through Gaussian process pr… ▽ More Spatiotemporal data analysis with massive zeros is widely used in many areas such as epidemiology and public health. We use a Bayesian framework to fit zero-inflated negative binomial models and employ a set of latent variables from Pólya-Gamma distributions to derive an efficient Gibbs sampler. The proposed model accommodates varying spatial and temporal random effects through Gaussian process priors, which have both the simplicity and flexibility in modeling nonlinear relationships through a covariance function. To conquer the computation bottleneck that GPs may suffer when the sample size is large, we adopt the nearest-neighbor GP approach that approximates the covariance matrix using local experts. For the simulation study, we adopt multiple settings with varying sizes of spatial locations to evaluate the performance of the proposed model such as spatial and temporal random effects estimation and compare the result to other methods. We also apply the proposed model to the COVID-19 death counts in the state of Florida, USA from 3/25/2020 through 7/29/2020 to examine relationships between social vulnerability and COVID-19 deaths. △ Less

Submitted 6 February, 2024; originally announced February 2024.

Journal ref: Journal of Statistical Planning and Inference (2024). 229, 106098

DARTS: Diffusion Approximated Residual Time Sampling for Time-of-flight Rendering in Homogeneous Scattering Media

Authors: Qianyue He, Dongyu Du, Haitian Jiang, Xin Jin

Abstract: Time-of-flight (ToF) devices have greatly propelled the advancement of various multi-modal perception applications. However, achieving accurate rendering of time-resolved information remains a challenge, particularly in scenes involving complex geometries, diverse materials and participating media. Existing ToF rendering works have demonstrated notable results, yet they struggle with scenes involv… ▽ More Time-of-flight (ToF) devices have greatly propelled the advancement of various multi-modal perception applications. However, achieving accurate rendering of time-resolved information remains a challenge, particularly in scenes involving complex geometries, diverse materials and participating media. Existing ToF rendering works have demonstrated notable results, yet they struggle with scenes involving scattering media and camera-warped settings. Other steady-state volumetric rendering methods exhibit significant bias or variance when directly applied to ToF rendering tasks. To address these challenges, we integrate transient diffusion theory into path construction and propose novel sampling methods for free-path distance and scattering direction, via resampled importance sampling and offline tabulation. An elliptical sampling method is further adapted to provide controllable vertex connection satisfying any required photon traversal time. In contrast to the existing temporal uniform sampling strategy, our method is the first to consider the contribution of transient radiance to importance-sample the full path, and thus enables improved temporal path construction under multiple scattering settings. The proposed method can be integrated into both path tracing and photon-based frameworks, delivering significant improvements in quality and efficiency with at least a 5x MSE reduction versus SOTA methods in equal rendering time. △ Less

Submitted 14 June, 2024; v1 submitted 5 February, 2024; originally announced February 2024.

Comments: 15 pages, 12 figures

ACM Class: I.3.7

High-fugacity expansion and crystallization in non-sliding hard-core lattice particle models without a tiling constraint

Authors: Qidong He, Ian Jauslin

Abstract: In this paper, we prove the existence of a crystallization transition for a family of hard-core particle models on periodic graphs in arbitrary dimensions. We establish a criterion under which crystallization occurs at sufficiently high densities. The criterion is more general than that in [Jauslin, Lebowitz, Comm. Math. Phys. 364:2, 2018], as it allows models in which particles do not tile the sp… ▽ More In this paper, we prove the existence of a crystallization transition for a family of hard-core particle models on periodic graphs in arbitrary dimensions. We establish a criterion under which crystallization occurs at sufficiently high densities. The criterion is more general than that in [Jauslin, Lebowitz, Comm. Math. Phys. 364:2, 2018], as it allows models in which particles do not tile the space in the close-packing configurations, such as discrete hard-disk models. To prove crystallization, we prove that the pressure is analytic in the inverse of the fugacity for large enough complex fugacities, using Pirogov-Sinai theory. One of the main tools used for this result is the definition of a local density, based on a discrete generalization of Voronoi cells. We illustrate the criterion by proving that it applies to two examples: staircase models and the radius 2.5 hard-disk model on the square lattice. △ Less

Submitted 4 February, 2024; originally announced February 2024.

Comments: 41 pages, 12 figures

Optimal-Landmark-Guided Image Blending for Face Morphing Attacks

Authors: Qiaoyun He, Zongyong Deng, Zuyuan He, Qijun Zhao

Abstract: In this paper, we propose a novel approach for conducting face morphing attacks, which utilizes optimal-landmark-guided image blending. Current face morphing attacks can be categorized into landmark-based and generation-based approaches. Landmark-based methods use geometric transformations to warp facial regions according to averaged landmarks but often produce morphed images with poor visual qual… ▽ More In this paper, we propose a novel approach for conducting face morphing attacks, which utilizes optimal-landmark-guided image blending. Current face morphing attacks can be categorized into landmark-based and generation-based approaches. Landmark-based methods use geometric transformations to warp facial regions according to averaged landmarks but often produce morphed images with poor visual quality. Generation-based methods, which employ generation models to blend multiple face images, can achieve better visual quality but are often unsuccessful in generating morphed images that can effectively evade state-of-the-art face recognition systems~(FRSs). Our proposed method overcomes the limitations of previous approaches by optimizing the morphing landmarks and using Graph Convolutional Networks (GCNs) to combine landmark and appearance features. We model facial landmarks as nodes in a bipartite graph that is fully connected and utilize GCNs to simulate their spatial and structural relationships. The aim is to capture variations in facial shape and enable accurate manipulation of facial appearance features during the warping process, resulting in morphed facial images that are highly realistic and visually faithful. Experiments on two public datasets prove that our method inherits the advantages of previous landmark-based and generation-based methods and generates morphed images with higher quality, posing a more significant threat to state-of-the-art FRSs. △ Less

Submitted 29 January, 2024; originally announced January 2024.

Comments: Accepted by IJCB2023

Choroidal thinning assessment through facial video analysis

Authors: Qinghua He, Yi Zhang, Mengxi Shen, Giovanni Gregori, Philip J. Rosenfeld, Ruikang K. Wang

Abstract: Different features of skin are associated with various medical conditions and provide opportunities to evaluate and monitor body health. This study created a strategy to assess choroidal thinning through the video analysis of facial skin. Videos capturing the entire facial skin were collected from 48 participants with age-related macular degeneration (AMD) and 12 healthy individuals. These facial… ▽ More Different features of skin are associated with various medical conditions and provide opportunities to evaluate and monitor body health. This study created a strategy to assess choroidal thinning through the video analysis of facial skin. Videos capturing the entire facial skin were collected from 48 participants with age-related macular degeneration (AMD) and 12 healthy individuals. These facial videos were analyzed using video-based trans-angiosomes imaging photoplethysmography (TaiPPG) to generate facial imaging biomarkers that were correlated with choroidal thickness (CT) measurements. The CT of all patients was determined using swept-source optical coherence tomography (SS-OCT). The results revealed the relationship between relative blood pulsation amplitude (BPA) in three typical facial angiosomes (cheek, side-forehead and mid-forehead) and the average macular CT (r = 0.48, p < 0.001; r = -0.56, p < 0.001; r = -0.40, p < 0.01). When considering a diagnostic threshold of 200μm, the newly developed facial video analysis tool effectively distinguished between cases of choroidal thinning and normal cases, yielding areas under the curve of 0.75, 0.79 and 0.69. These findings shed light on the connection between choroidal blood flow and facial skin hemodynamics, which suggests the potential for predicting vascular diseases through widely accessible skin imaging data. △ Less

Submitted 29 January, 2024; originally announced January 2024.

Comments: 8 pages, 4 figures

Higher-order topology in Fibonacci quasicrystals

Authors: Chaozhi Ouyang, Qinghua He, Dong-Hui Xu, Feng Liu

Abstract: In crystalline systems, higher-order topology, characterized by topological states of codimension greater than one, typically arises from the mismatch between Wannier centers and atomic sites, leading to filling anomalies. However, this phenomenon is less understood in aperiodic systems, such as quasicrystals, where Wannier centers are not well defined. In this study, we examine Fibonacci chains a… ▽ More In crystalline systems, higher-order topology, characterized by topological states of codimension greater than one, typically arises from the mismatch between Wannier centers and atomic sites, leading to filling anomalies. However, this phenomenon is less understood in aperiodic systems, such as quasicrystals, where Wannier centers are not well defined. In this study, we examine Fibonacci chains and squares, a quintessential type of quasicrystal, to investigate their higher-order topological properties. We discover that topological interfacial states, including corner states, can be inherited from their higher-dimensional periodic counterparts, such as the two-dimensional Su-Schrieffer-Heeger model. This finding is validated through numerical simulations of both phononic and photonic Fibonacci quasicrystals by the finite element method, revealing the emergence of topological edge and corner states at interfaces between Fibonacci quasicrystals with differing topologies inherited from their parent systems. Our results not only provide insight into the higher-order topology of quasicrystals but also open avenues for exploring novel topological phases in aperiodic structures. △ Less

Submitted 26 January, 2024; originally announced January 2024.

Comments: 9 pages, 6 figures

FairSample: Training Fair and Accurate Graph Convolutional Neural Networks Efficiently

Authors: Zicun Cong, Shi Baoxu, Shan Li, Jaewon Yang, Qi He, Jian Pei

Abstract: Fairness in Graph Convolutional Neural Networks (GCNs) becomes a more and more important concern as GCNs are adopted in many crucial applications. Societal biases against sensitive groups may exist in many real world graphs. GCNs trained on those graphs may be vulnerable to being affected by such biases. In this paper, we adopt the well-known fairness notion of demographic parity and tackle the ch… ▽ More Fairness in Graph Convolutional Neural Networks (GCNs) becomes a more and more important concern as GCNs are adopted in many crucial applications. Societal biases against sensitive groups may exist in many real world graphs. GCNs trained on those graphs may be vulnerable to being affected by such biases. In this paper, we adopt the well-known fairness notion of demographic parity and tackle the challenge of training fair and accurate GCNs efficiently. We present an in-depth analysis on how graph structure bias, node attribute bias, and model parameters may affect the demographic parity of GCNs. Our insights lead to FairSample, a framework that jointly mitigates the three types of biases. We employ two intuitive strategies to rectify graph structures. First, we inject edges across nodes that are in different sensitive groups but similar in node features. Second, to enhance model fairness and retain model quality, we develop a learnable neighbor sampling policy using reinforcement learning. To address the bias in node features and model parameters, FairSample is complemented by a regularization objective to optimize fairness. △ Less

Submitted 26 January, 2024; originally announced January 2024.

Comments: Accepted by TKDE 2023

UniM-OV3D: Uni-Modality Open-Vocabulary 3D Scene Understanding with Fine-Grained Feature Representation

Authors: Qingdong He, Jinlong Peng, Zhengkai Jiang, Kai Wu, Xiaozhong Ji, Jiangning Zhang, Yabiao Wang, Chengjie Wang, Mingang Chen, Yunsheng Wu

Abstract: 3D open-vocabulary scene understanding aims to recognize arbitrary novel categories beyond the base label space. However, existing works not only fail to fully utilize all the available modal information in the 3D domain but also lack sufficient granularity in representing the features of each modality. In this paper, we propose a unified multimodal 3D open-vocabulary scene understanding network,… ▽ More 3D open-vocabulary scene understanding aims to recognize arbitrary novel categories beyond the base label space. However, existing works not only fail to fully utilize all the available modal information in the 3D domain but also lack sufficient granularity in representing the features of each modality. In this paper, we propose a unified multimodal 3D open-vocabulary scene understanding network, namely UniM-OV3D, which aligns point clouds with image, language and depth. To better integrate global and local features of the point clouds, we design a hierarchical point cloud feature extraction module that learns comprehensive fine-grained feature representations. Further, to facilitate the learning of coarse-to-fine point-semantic representations from captions, we propose the utilization of hierarchical 3D caption pairs, capitalizing on geometric constraints across various viewpoints of 3D scenes. Extensive experimental results demonstrate the effectiveness and superiority of our method in open-vocabulary semantic and instance segmentation, which achieves state-of-the-art performance on both indoor and outdoor benchmarks such as ScanNet, ScanNet200, S3IDS and nuScenes. Code is available at https://github.com/hithqd/UniM-OV3D. △ Less

Submitted 20 April, 2024; v1 submitted 20 January, 2024; originally announced January 2024.

Comments: Accepted by IJCAI 2024

Ultra-lightweight Neural Differential DSP Vocoder For High Quality Speech Synthesis

Authors: Prabhav Agrawal, Thilo Koehler, Zhiping Xiu, Prashant Serai, Qing He

Abstract: Neural vocoders model the raw audio waveform and synthesize high-quality audio, but even the highly efficient ones, like MB-MelGAN and LPCNet, fail to run real-time on a low-end device like a smartglass. A pure digital signal processing (DSP) based vocoder can be implemented via lightweight fast Fourier transforms (FFT), and therefore, is a magnitude faster than any neural vocoder. A DSP vocoder o… ▽ More Neural vocoders model the raw audio waveform and synthesize high-quality audio, but even the highly efficient ones, like MB-MelGAN and LPCNet, fail to run real-time on a low-end device like a smartglass. A pure digital signal processing (DSP) based vocoder can be implemented via lightweight fast Fourier transforms (FFT), and therefore, is a magnitude faster than any neural vocoder. A DSP vocoder often gets a lower audio quality due to consuming over-smoothed acoustic model predictions of approximate representations for the vocal tract. In this paper, we propose an ultra-lightweight differential DSP (DDSP) vocoder that uses a jointly optimized acoustic model with a DSP vocoder, and learns without an extracted spectral feature for the vocal tract. The model achieves audio quality comparable to neural vocoders with a high average MOS of 4.36 while being efficient as a DSP vocoder. Our C++ implementation, without any hardware-specific optimization, is at 15 MFLOPS, surpasses MB-MelGAN by 340 times in terms of FLOPS, and achieves a vocoder-only RTF of 0.003 and overall RTF of 0.044 while running single-threaded on a 2GHz Intel Xeon CPU. △ Less

Submitted 18 January, 2024; originally announced January 2024.

Comments: Accepted for ICASSP 2024

Role of Elastic Phonon Couplings in Dictating the Thermal Transport across Atomically Sharp SiC/Si Interfaces

Authors: Qinqin He, Yixin Xu, Haidong Wang, Zhigang Li, Yanguang Zhou

Abstract: Wide-bandgap (WBG) semiconductors have promising applications in power electronics due to their high voltages, radio frequencies, and tolerant temperatures. Among all the WBG semiconductors, SiC has attracted attention because of its high mobility, high thermal stability, and high thermal conductivity. However, the interfaces between SiC and the corresponding substrate largely affect the performan… ▽ More Wide-bandgap (WBG) semiconductors have promising applications in power electronics due to their high voltages, radio frequencies, and tolerant temperatures. Among all the WBG semiconductors, SiC has attracted attention because of its high mobility, high thermal stability, and high thermal conductivity. However, the interfaces between SiC and the corresponding substrate largely affect the performance of SiC-based electronics. It is therefore necessary to understand and design the interfacial thermal transport across the SiC/substrate interfaces, which is critical for the thermal management design of these SiC-based power electronics. This work systematically investigates heat transfer across the 3C-SiC/Si, 4H-SiC/Si, and 6H-SiC/Si interfaces using non-equilibrium molecular dynamics simulations and diffuse mismatch model. We find that the room temperature ITC for 3C-SiC/Si, 4H-SiC/Si, and 6H-SiC/Si interfaces is 932 MW/m2K, 759 MW/m2K, and 697 MW/m2K, respectively. We also show the contribution of the ITC resulting from elastic scatterings at room temperature is 80% for 3C-SiC/Si interfaces, 85% for 4H-SiC/Si interfaces, and 82% for 6H-SiC/Si interfaces, respectively. We further find the ITC contributed by the elastic scattering decreases with the temperature but remains at a high ratio of 67%~78% even at an ultrahigh temperature of 1000 K. The reason for such a high elastic ITC is the large overlap between the vibrational density of states of Si and SiC at low frequencies (< ~ 18 THz), which is also demonstrated by the diffuse mismatch mode. It is interesting to find that the inelastic ITC resulting from the phonons with frequencies higher than the cutoff frequency of Si (i.e., ~18 THz) can be negligible. That may be because of the wide frequency gap between Si and SiC, which makes the inelastic scattering among these phonons challenging to meet the energy and momentum conservation rules. △ Less

Submitted 17 January, 2024; originally announced January 2024.

Small Language Model Can Self-correct

Authors: Haixia Han, Jiaqing Liang, Jie Shi, Qianyu He, Yanghua Xiao

Abstract: Generative Language Models (LMs) such as ChatGPT have exhibited remarkable performance across various downstream tasks. Nevertheless, one of their most prominent drawbacks is generating inaccurate or false information with a confident tone. Previous studies have devised sophisticated pipelines and prompts to induce large LMs to exhibit the capability for self-correction. However, large LMs are exp… ▽ More Generative Language Models (LMs) such as ChatGPT have exhibited remarkable performance across various downstream tasks. Nevertheless, one of their most prominent drawbacks is generating inaccurate or false information with a confident tone. Previous studies have devised sophisticated pipelines and prompts to induce large LMs to exhibit the capability for self-correction. However, large LMs are explicitly prompted to verify and modify its answers separately rather than completing all steps spontaneously like humans. Moreover, these complex prompts are extremely challenging for small LMs to follow. In this paper, we introduce the \underline{I}ntrinsic \underline{S}elf-\underline{C}orrection (ISC) in generative language models, aiming to correct the initial output of LMs in a self-triggered manner, even for those small LMs with 6 billion parameters. Specifically, we devise a pipeline for constructing self-correction data and propose Partial Answer Masking (PAM), aiming to endow the model with the capability for intrinsic self-correction through fine-tuning. We conduct experiments using LMs with parameters sizes ranging from 6 billion to 13 billion in two tasks, including commonsense reasoning and factual knowledge reasoning. Our experiments demonstrate that the outputs generated using ISC outperform those generated without self-correction. We believe that the output quality of even small LMs can be further improved by empowering them with the ability to intrinsic self-correct. △ Less

Submitted 11 May, 2024; v1 submitted 14 January, 2024; originally announced January 2024.

Multi-Task Learning for Front-End Text Processing in TTS

Authors: Wonjune Kang, Yun Wang, Shun Zhang, Arthur Hinsvark, Qing He

Abstract: We propose a multi-task learning (MTL) model for jointly performing three tasks that are commonly solved in a text-to-speech (TTS) front-end: text normalization (TN), part-of-speech (POS) tagging, and homograph disambiguation (HD). Our framework utilizes a tree-like structure with a trunk that learns shared representations, followed by separate task-specific heads. We further incorporate a pre-tra… ▽ More We propose a multi-task learning (MTL) model for jointly performing three tasks that are commonly solved in a text-to-speech (TTS) front-end: text normalization (TN), part-of-speech (POS) tagging, and homograph disambiguation (HD). Our framework utilizes a tree-like structure with a trunk that learns shared representations, followed by separate task-specific heads. We further incorporate a pre-trained language model to utilize its built-in lexical and contextual knowledge, and study how to best use its embeddings so as to most effectively benefit our multi-task model. Through task-wise ablations, we show that our full model trained on all three tasks achieves the strongest overall performance compared to models trained on individual or sub-combinations of tasks, confirming the advantages of our MTL framework. Finally, we introduce a new HD dataset containing a balanced number of sentences in diverse contexts for a variety of homographs and their pronunciations. We demonstrate that incorporating this dataset into training significantly improves HD performance over only using a commonly used, but imbalanced, pre-existing dataset. △ Less

Submitted 11 January, 2024; originally announced January 2024.

Comments: ICASSP 2024

Amplification of genuine tripartite nonlocality and entanglement in the Schwarzschild spacetime under decoherence

Authors: Chunyao Liu, Zhengwen Long, Qiliang He

Abstract: We investigate the amplification of the genuine tripartite nonlocality(GTN) and the genuine tripartite entanglement(GTE) of Dirac particles in the background of a Schwarzschild black hole by a local filtering operation under decoherence. It is shown that the physically accessible GTN will be completely destroyed by decoherence, which means that the physically accessible GTN will not exist in the s… ▽ More We investigate the amplification of the genuine tripartite nonlocality(GTN) and the genuine tripartite entanglement(GTE) of Dirac particles in the background of a Schwarzschild black hole by a local filtering operation under decoherence. It is shown that the physically accessible GTN will be completely destroyed by decoherence, which means that the physically accessible GTN will not exist in the system. Particularly, the local filtering operation can make the physically accessible GTN appear within a certain range of Hawking temperature, namely, the local filtering operation can cause the physically accessible GTN to be generated in the system coupled with the environment, which is not discovered before and is benefit for the quantum information processing. Furthermore, we also find that the physically accessible GTE approaches a stable value in the limit of infinite Hawking temperature for most cases, but if the decoherence parameter $p$ is less than 1, the ``sudden death'' of GTE will take place when the decoherence strength is large enough. It is worth noting that the nonzero stable value of GTE can be increased by performing the local filtering operation, even in the presence of decoherence. Finally, we explore the generation of physically inaccessible GTN and GTE of other tripartite subsystems under decoherence, it is shown that the physically inaccessible GTN cannot be produced, but the physically inaccessible GTE can be produced. In addition, we can see that the generated physically inaccessible GTE can be increased by applying the local filtering operation. △ Less

Submitted 20 May, 2024; v1 submitted 9 January, 2024; originally announced January 2024.

Comments: 11 pages, 9 figures. arXiv admin note: text overlap with arXiv:2201.02333 by other authors

SonicVisionLM: Playing Sound with Vision Language Models

Authors: Zhifeng Xie, Shengye Yu, Qile He, Mengtian Li

Abstract: There has been a growing interest in the task of generating sound for silent videos, primarily because of its practicality in streamlining video post-production. However, existing methods for video-sound generation attempt to directly create sound from visual representations, which can be challenging due to the difficulty of aligning visual representations with audio representations. In this paper… ▽ More There has been a growing interest in the task of generating sound for silent videos, primarily because of its practicality in streamlining video post-production. However, existing methods for video-sound generation attempt to directly create sound from visual representations, which can be challenging due to the difficulty of aligning visual representations with audio representations. In this paper, we present SonicVisionLM, a novel framework aimed at generating a wide range of sound effects by leveraging vision-language models(VLMs). Instead of generating audio directly from video, we use the capabilities of powerful VLMs. When provided with a silent video, our approach first identifies events within the video using a VLM to suggest possible sounds that match the video content. This shift in approach transforms the challenging task of aligning image and audio into more well-studied sub-problems of aligning image-to-text and text-to-audio through the popular diffusion models. To improve the quality of audio recommendations with LLMs, we have collected an extensive dataset that maps text descriptions to specific sound effects and developed a time-controlled audio adapter. Our approach surpasses current state-of-the-art methods for converting video to audio, enhancing synchronization with the visuals, and improving alignment between audio and video components. Project page: https://yusiissy.github.io/SonicVisionLM.github.io/ △ Less

Submitted 3 April, 2024; v1 submitted 9 January, 2024; originally announced January 2024.

Comments: CVPR 2024

Deep FBSDE Neural Networks for Solving Incompressible Navier-Stokes Equation and Cahn-Hilliard Equation

Authors: Yangtao Deng, Qiaolin He

Abstract: Efficient algorithms for solving high-dimensional partial differential equations (PDEs) has been an exceedingly difficult task for a long time, due to the curse of dimensionality. We extend the forward-backward stochastic neural networks (FBSNNs) which depends on forward-backward stochastic differential equation (FBSDE) to solve incompressible Navier-Stokes equation. For Cahn-Hilliard equation, we… ▽ More Efficient algorithms for solving high-dimensional partial differential equations (PDEs) has been an exceedingly difficult task for a long time, due to the curse of dimensionality. We extend the forward-backward stochastic neural networks (FBSNNs) which depends on forward-backward stochastic differential equation (FBSDE) to solve incompressible Navier-Stokes equation. For Cahn-Hilliard equation, we derive a modified Cahn-Hilliard equation from a widely used stabilized scheme for original Cahn-Hilliard equation. This equation can be written as a continuous parabolic system, where FBSDE can be applied and the unknown solution is approximated by neural network. Also our method is successfully developed to Cahn-Hilliard-Navier-Stokes (CHNS) equation. The accuracy and stability of our methods are shown in many numerical experiments, specially in high dimension. △ Less

Submitted 19 June, 2024; v1 submitted 7 January, 2024; originally announced January 2024.

Large Language Models for Social Networks: Applications, Challenges, and Solutions

Authors: Jingying Zeng, Richard Huang, Waleed Malik, Langxuan Yin, Bojan Babic, Danny Shacham, Xiao Yan, Jaewon Yang, Qi He

Abstract: Large Language Models (LLMs) are transforming the way people generate, explore, and engage with content. We study how we can develop LLM applications for online social networks. Despite LLMs' successes in other domains, it is challenging to develop LLM-based products for social networks for numerous reasons, and it has been relatively under-reported in the research community. We categorize LLM app… ▽ More Large Language Models (LLMs) are transforming the way people generate, explore, and engage with content. We study how we can develop LLM applications for online social networks. Despite LLMs' successes in other domains, it is challenging to develop LLM-based products for social networks for numerous reasons, and it has been relatively under-reported in the research community. We categorize LLM applications for social networks into three categories. First is knowledge tasks where users want to find new knowledge and information, such as search and question-answering. Second is entertainment tasks where users want to consume interesting content, such as getting entertaining notification content. Third is foundational tasks that need to be done to moderate and operate the social networks, such as content annotation and LLM monitoring. For each task, we share the challenges we found, solutions we developed, and lessons we learned. To the best of our knowledge, this is the first comprehensive paper about developing LLM applications for social networks. △ Less

Submitted 4 January, 2024; originally announced January 2024.

Design of a full-filed transmission X-ray microscope with 30nm resolution

Authors: Keliang Liao, Qili He, Panyun Li, Maohua Song, Peiping Zhu

Abstract: A full-field transmission hard X-ray microscope (TXM) with 30nm resolution was designed and its prototype was constructed. The TXM relies on a compact, high stiffness, low heat dissipation and low vibration design philosophy and utilizes Fresnel Zone plate (FZP) as imaging optics. The design of the TXM was introduced in detail, including the optical layout, the parameters of the FZP, the mechanica… ▽ More A full-field transmission hard X-ray microscope (TXM) with 30nm resolution was designed and its prototype was constructed. The TXM relies on a compact, high stiffness, low heat dissipation and low vibration design philosophy and utilizes Fresnel Zone plate (FZP) as imaging optics. The design of the TXM was introduced in detail, including the optical layout, the parameters of the FZP, the mechanical design of the TXM instrument. Preliminary imaging result with 52nm spatial resolution was achieved. △ Less

Submitted 15 November, 2023; originally announced January 2024.

Enhancing Quantitative Reasoning Skills of Large Language Models through Dimension Perception

Authors: Yuncheng Huang, Qianyu He, Jiaqing Liang, Sihang Jiang, Yanghua Xiao, Yunwen Chen

Abstract: Quantities are distinct and critical components of texts that characterize the magnitude properties of entities, providing a precise perspective for the understanding of natural language, especially for reasoning tasks. In recent years, there has been a flurry of research on reasoning tasks based on large language models (LLMs), most of which solely focus on numerical values, neglecting the dimens… ▽ More Quantities are distinct and critical components of texts that characterize the magnitude properties of entities, providing a precise perspective for the understanding of natural language, especially for reasoning tasks. In recent years, there has been a flurry of research on reasoning tasks based on large language models (LLMs), most of which solely focus on numerical values, neglecting the dimensional concept of quantities with units despite its importance. We argue that the concept of dimension is essential for precisely understanding quantities and of great significance for LLMs to perform quantitative reasoning. However, the lack of dimension knowledge and quantity-related benchmarks has resulted in low performance of LLMs. Hence, we present a framework to enhance the quantitative reasoning ability of language models based on dimension perception. We first construct a dimensional unit knowledge base (DimUnitKB) to address the knowledge gap in this area. We propose a benchmark DimEval consisting of seven tasks of three categories to probe and enhance the dimension perception skills of LLMs. To evaluate the effectiveness of our methods, we propose a quantitative reasoning task and conduct experiments. The experimental results show that our dimension perception method dramatically improves accuracy (43.55%->50.67%) on quantitative reasoning tasks compared to GPT-4. △ Less

Submitted 29 December, 2023; originally announced December 2023.

Comments: Accepted in the 40th IEEE International Conference on Data Engineering (ICDE 2024)

On the Kudla-Rapoport conjecture for unitary Shimura varieties with maximal parahoric level structure at unramified primes

Authors: Sungyoon Cho, Qiao He, Zhiyu Zhang

Abstract: In this article, we prove that a version of Tate conjectures for certain Deligne-Lusztig varieties implies the Kudla-Rapoport conjecture for unitary Shimura varieties with maximal parahoric level at unramified primes. Furthermore, we prove that the Kudla-Rapoport conjecture holds unconditionally for several new cases in any dimension. In this article, we prove that a version of Tate conjectures for certain Deligne-Lusztig varieties implies the Kudla-Rapoport conjecture for unitary Shimura varieties with maximal parahoric level at unramified primes. Furthermore, we prove that the Kudla-Rapoport conjecture holds unconditionally for several new cases in any dimension. △ Less

Submitted 28 December, 2023; originally announced December 2023.

Comments: 93 pages

Incompressible limit of porous media equation with chemotaxis and growth

Authors: Qingyou He, Hai-Liang Li, Benoît Perthame

Abstract: We revisit the problem of proving the incompressible limit for the compressible porous media equation with Newtonian drift and growth. The question is motivated by models of living tissues development including chemotaxis. We extend the problem, already treated by the authors and several other contributions, in using a simplified approach, in treating dimensions two or higher, and in incorporating… ▽ More We revisit the problem of proving the incompressible limit for the compressible porous media equation with Newtonian drift and growth. The question is motivated by models of living tissues development including chemotaxis. We extend the problem, already treated by the authors and several other contributions, in using a simplified approach, in treating dimensions two or higher, and in incorporating the pressure driven growth term. We also complete the analysis with stronger $L^4$ estimates on the pressure gradient. The major difficulty is to prove the strong convergence of the pressure gradient which is obtained here by a new observation on an algebraic relation involving the pressure gradient for weak limits. △ Less

Submitted 28 December, 2023; originally announced December 2023.

DreamTuner: Single Image is Enough for Subject-Driven Generation

Authors: Miao Hua, Jiawei Liu, Fei Ding, Wei Liu, Jie Wu, Qian He

Abstract: Diffusion-based models have demonstrated impressive capabilities for text-to-image generation and are expected for personalized applications of subject-driven generation, which require the generation of customized concepts with one or a few reference images. However, existing methods based on fine-tuning fail to balance the trade-off between subject learning and the maintenance of the generation c… ▽ More Diffusion-based models have demonstrated impressive capabilities for text-to-image generation and are expected for personalized applications of subject-driven generation, which require the generation of customized concepts with one or a few reference images. However, existing methods based on fine-tuning fail to balance the trade-off between subject learning and the maintenance of the generation capabilities of pretrained models. Moreover, other methods that utilize additional image encoders tend to lose important details of the subject due to encoding compression. To address these challenges, we propose DreamTurner, a novel method that injects reference information from coarse to fine to achieve subject-driven image generation more effectively. DreamTurner introduces a subject-encoder for coarse subject identity preservation, where the compressed general subject features are introduced through an attention layer before visual-text cross-attention. We then modify the self-attention layers within pretrained text-to-image models to self-subject-attention layers to refine the details of the target subject. The generated image queries detailed features from both the reference image and itself in self-subject-attention. It is worth emphasizing that self-subject-attention is an effective, elegant, and training-free method for maintaining the detailed features of customized subjects and can serve as a plug-and-play solution during inference. Finally, with additional subject-driven fine-tuning, DreamTurner achieves remarkable performance in subject-driven image generation, which can be controlled by a text or other conditions such as pose. For further details, please visit the project page at https://dreamtuner-diffusion.github.io/. △ Less

Submitted 21 December, 2023; originally announced December 2023.

Let AI Entertain You: Increasing User Engagement with Generative AI and Rejection Sampling

Authors: Jingying Zeng, Jaewon Yang, Waleed Malik, Xiao Yan, Richard Huang, Qi He

Abstract: While generative AI excels in content generation, it does not always increase user engagement. This can be attributed to two main factors. First, generative AI generates content without incorporating explicit or implicit feedback about user interactions. Even if the generated content seems to be more informative or well-written, it does not necessarily lead to an increase in user activities, such… ▽ More While generative AI excels in content generation, it does not always increase user engagement. This can be attributed to two main factors. First, generative AI generates content without incorporating explicit or implicit feedback about user interactions. Even if the generated content seems to be more informative or well-written, it does not necessarily lead to an increase in user activities, such as clicks. Second, there is a concern with the quality of the content generative AI produces, which often lacks the distinctiveness and authenticity that human-created content possesses. These two factors can lead to content that fails to meet specific needs and preferences of users, ultimately reducing its potential to be engaging. This paper presents a generic framework of how to improve user engagement with generative AI by leveraging user feedback. Our solutions employ rejection sampling, a technique used in reinforcement learning, to boost engagement metrics. We leveraged the framework in the context of email notification subject lines generation for an online social network, and achieved significant engagement metric lift including +1% Session and +0.4% Weekly Active Users. We believe our work offers a universal framework that enhances user engagement with generative AI, particularly when standard generative AI reaches its limits in terms of enhancing content to be more captivating. To the best of our knowledge, this represents an early milestone in the industry's successful use of generative AI to enhance user engagement. △ Less

Submitted 16 December, 2023; originally announced December 2023.

The Galactic distribution of pulsar scattering and the $τ-{\rm DM}$ relation

Authors: Qiuyi He, Xun Shi

Abstract: Interstellar radio wave scattering leads to flux density fluctuations and pulse broadening of pulsar signals. However, Galactic distribution and the structure of the scattering medium are still poorly understood. Pulsar pulse broadening data available for a relatively large number of pulsars is well suited for such investigations. We collected an up-to-date sample of publicly available pulsar scat… ▽ More Interstellar radio wave scattering leads to flux density fluctuations and pulse broadening of pulsar signals. However, Galactic distribution and the structure of the scattering medium are still poorly understood. Pulsar pulse broadening data available for a relatively large number of pulsars is well suited for such investigations. We collected an up-to-date sample of publicly available pulsar scattering data and introduced a new quantity -- the reduced scattering strength $\tildeτ$ to study the Galactic distribution of pulsar scattering in the Milky Way. We show that the current observations are dominated by two distinct pulsar populations: a local and an inner-Galactic one separated by $\tilde{τ}=10^{-5.1}\,{\rm s}\,{\rm cm}^{6}\,{\rm pc}^{-1}$. The stronger electron density fluctuations associated with the inner-Galactic population naturally explain the observed steepening of pulsar scattering time $τ$ - dispersion measure relation. We measure an inner disc region with $3\,{\rm kpc}<\rm r< 5.5\,{\rm kpc}$ from the Galactic centre to have a scattering scale height of about $0.28\,{\rm kpc}$, supporting a correlation between interstellar radio scattering and structures associating with the ionized gas and stellar activities. △ Less

Submitted 18 December, 2023; originally announced December 2023.

Journal ref: Mon. Not. R. Astron. Soc. 527, 5183 (2024)

Single PW takes a shortcut to compound PW in US imaging

Authors: Zhiqiang Li, Hengrong Lan, Lijie Huang, Qiong He, Jianwen Luo

Abstract: Reconstruction of ultrasound (US) images from radio-frequency data can be conceptualized as a linear inverse problem. Traditional deep learning approaches, which aim to improve the quality of US images by directly learning priors, often encounter challenges in generalization. Recently, diffusion-based generative models have received significant attention within the research community due to their… ▽ More Reconstruction of ultrasound (US) images from radio-frequency data can be conceptualized as a linear inverse problem. Traditional deep learning approaches, which aim to improve the quality of US images by directly learning priors, often encounter challenges in generalization. Recently, diffusion-based generative models have received significant attention within the research community due to their robust performance in image reconstruction tasks. However, a limitation of these models is their inherent low speed in generating image samples from pure Gaussian noise progressively. In this study, we exploit the inherent similarity between the US images reconstructed from a single plane wave (PW) and PW compounding PWC). We hypothesize that a single PW can take a shortcut to reach the diffusion trajectory of PWC, removing the need to begin with Gaussian noise. By employing an advanced diffusion model, we demonstrate its effectiveness in US image reconstruction, achieving a substantial reduction in sampling steps. In-vivo experimental results indicate that our approach can reduce sampling steps by 60%, while preserving comparable performance metrics with the conventional diffusion model. △ Less

Submitted 14 December, 2023; originally announced December 2023.

Comments: Submitted to ISBI 2024

Fast Sampling generative model for Ultrasound image reconstruction

Authors: Hengrong Lan, Zhiqiang Li, Qiong He, Jianwen Luo

Abstract: Image reconstruction from radio-frequency data is pivotal in ultrafast plane wave ultrasound imaging. Unlike the conventional delay-and-sum (DAS) technique, which relies on somewhat imprecise assumptions, deep learning-based methods perform image reconstruction by training on paired data, leading to a notable enhancement in image quality. Nevertheless, these strategies often exhibit limited genera… ▽ More Image reconstruction from radio-frequency data is pivotal in ultrafast plane wave ultrasound imaging. Unlike the conventional delay-and-sum (DAS) technique, which relies on somewhat imprecise assumptions, deep learning-based methods perform image reconstruction by training on paired data, leading to a notable enhancement in image quality. Nevertheless, these strategies often exhibit limited generalization capabilities. Recently, denoising diffusion models have become the preferred paradigm for image reconstruction tasks. However, their reliance on an iterative sampling procedure results in prolonged generation time. In this paper, we propose a novel sampling framework that concurrently enforces data consistency of ultrasound signals and data-driven priors. By leveraging the advanced diffusion model, the generation of high-quality images is substantially expedited. Experimental evaluations on an in-vivo dataset indicate that our approach with a single plane wave surpasses DAS with spatial coherent compounding of 75 plane waves. △ Less

Submitted 14 December, 2023; originally announced December 2023.

Comments: submitted to ISBI 2024

MuFuzz: Sequence-Aware Mutation and Seed Mask Guidance for Blockchain Smart Contract Fuzzing

Authors: Peng Qian, Hanjie Wu, Zeren Du, Turan Vural, Dazhong Rong, Zheng Cao, Lun Zhang, Yanbin Wang, Jianhai Chen, Qinming He

Abstract: As blockchain smart contracts become more widespread and carry more valuable digital assets, they become an increasingly attractive target for attackers. Over the past few years, smart contracts have been subject to a plethora of devastating attacks, resulting in billions of dollars in financial losses. There has been a notable surge of research interest in identifying defects in smart contracts.… ▽ More As blockchain smart contracts become more widespread and carry more valuable digital assets, they become an increasingly attractive target for attackers. Over the past few years, smart contracts have been subject to a plethora of devastating attacks, resulting in billions of dollars in financial losses. There has been a notable surge of research interest in identifying defects in smart contracts. However, existing smart contract fuzzing tools are still unsatisfactory. They struggle to screen out meaningful transaction sequences and specify critical inputs for each transaction. As a result, they can only trigger a limited range of contract states, making it difficult to unveil complicated vulnerabilities hidden in the deep state space. In this paper, we shed light on smart contract fuzzing by employing a sequence-aware mutation and seed mask guidance strategy. In particular, we first utilize data-flow-based feedback to determine transaction orders in a meaningful way and further introduce a sequence-aware mutation technique to explore deeper states. Thereafter, we design a mask-guided seed mutation strategy that biases the generated transaction inputs to hit target branches. In addition, we develop a dynamic-adaptive energy adjustment paradigm that balances the fuzzing resource allocation during a fuzzing campaign. We implement our designs into a new smart contract fuzzer named MuFuzz, and extensively evaluate it on three benchmarks. Empirical results demonstrate that MuFuzz outperforms existing tools in terms of both branch coverage and bug finding. Overall, MuFuzz achieves higher branch coverage than state-of-the-art fuzzers (up to 25%) and detects 30% more bugs than existing bug detectors. △ Less

Submitted 9 December, 2023; v1 submitted 7 December, 2023; originally announced December 2023.

Comments: This paper has been accepted by ICDE 2024

Moving Sampling Physics-informed Neural Networks induced by Moving Mesh PDE

Authors: Yu Yang, Qihong Yang, Yangtao Deng, Qiaolin He

Abstract: In this work, we propose an end-to-end adaptive sampling neural network (MMPDE-Net) based on the moving mesh method, which can adaptively generate new sampling points by solving the moving mesh PDE. This model focuses on improving the quality of sampling points generation. Moreover, we develop an iterative algorithm based on MMPDE-Net, which makes the sampling points more precise and controllable.… ▽ More In this work, we propose an end-to-end adaptive sampling neural network (MMPDE-Net) based on the moving mesh method, which can adaptively generate new sampling points by solving the moving mesh PDE. This model focuses on improving the quality of sampling points generation. Moreover, we develop an iterative algorithm based on MMPDE-Net, which makes the sampling points more precise and controllable. Since MMPDE-Net is a framework independent of the deep learning solver, we combine it with physics-informed neural networks (PINN) to propose moving sampling PINN (MS-PINN) and demonstrate its effectiveness by error analysis under some assumptions. Finally, we demonstrate the performance improvement of MS-PINN compared to PINN through numerical experiments of four typical examples, which numerically verify the effectiveness of our method. △ Less

Submitted 9 June, 2024; v1 submitted 14 November, 2023; originally announced November 2023.

Neural-Integrated Meshfree (NIM) Method: A differentiable programming-based hybrid solver for computational mechanics

Authors: Honghui Du, QiZhi He

Abstract: We present the neural-integrated meshfree (NIM) method, a differentiable programming-based hybrid meshfree approach within the field of computational mechanics. NIM seamlessly integrates traditional physics-based meshfree discretization techniques with deep learning architectures. It employs a hybrid approximation scheme, NeuroPU, to effectively represent the solution by combining continuous DNN r… ▽ More We present the neural-integrated meshfree (NIM) method, a differentiable programming-based hybrid meshfree approach within the field of computational mechanics. NIM seamlessly integrates traditional physics-based meshfree discretization techniques with deep learning architectures. It employs a hybrid approximation scheme, NeuroPU, to effectively represent the solution by combining continuous DNN representations with partition of unity (PU) basis functions associated with the underlying spatial discretization. This neural-numerical hybridization not only enhances the solution representation through functional space decomposition but also reduces both the size of DNN model and the need for spatial gradient computations based on automatic differentiation, leading to a significant improvement in training efficiency. Under the NIM framework, we propose two truly meshfree solvers: the strong form-based NIM (S-NIM) and the local variational form-based NIM (V-NIM). In the S-NIM solver, the strong-form governing equation is directly considered in the loss function, while the V-NIM solver employs a local Petrov-Galerkin approach that allows the construction of variational residuals based on arbitrary overlapping subdomains. This ensures both the satisfaction of underlying physics and the preservation of meshfree property. We perform extensive numerical experiments on both stationary and transient benchmark problems to assess the effectiveness of the proposed NIM methods in terms of accuracy, scalability, generalizability, and convergence properties. Moreover, comparative analysis with other physics-informed machine learning methods demonstrates that NIM, especially V-NIM, significantly enhances both accuracy and efficiency in end-to-end predictive capabilities. △ Less

Submitted 21 November, 2023; originally announced November 2023.

Understanding Data Augmentation from a Robustness Perspective

Authors: Zhendong Liu, Jie Zhang, Qiangqiang He, Chongjun Wang

Abstract: In the realm of visual recognition, data augmentation stands out as a pivotal technique to amplify model robustness. Yet, a considerable number of existing methodologies lean heavily on heuristic foundations, rendering their intrinsic mechanisms ambiguous. This manuscript takes both a theoretical and empirical approach to understanding the phenomenon. Theoretically, we frame the discourse around d… ▽ More In the realm of visual recognition, data augmentation stands out as a pivotal technique to amplify model robustness. Yet, a considerable number of existing methodologies lean heavily on heuristic foundations, rendering their intrinsic mechanisms ambiguous. This manuscript takes both a theoretical and empirical approach to understanding the phenomenon. Theoretically, we frame the discourse around data augmentation within game theory's constructs. Venturing deeper, our empirical evaluations dissect the intricate mechanisms of emblematic data augmentation strategies, illuminating that these techniques primarily stimulate mid- and high-order game interactions. Beyond the foundational exploration, our experiments span multiple datasets and diverse augmentation techniques, underscoring the universal applicability of our findings. Recognizing the vast array of robustness metrics with intricate correlations, we unveil a streamlined proxy. This proxy not only simplifies robustness assessment but also offers invaluable insights, shedding light on the inherent dynamics of model game interactions and their relation to overarching system robustness. These insights provide a novel lens through which we can re-evaluate model safety and robustness in visual recognition tasks. △ Less

Submitted 7 September, 2023; originally announced November 2023.

Comments: Not published yet. arXiv admin note: text overlap with arXiv:2212.04059

Efficient Temporally-Aware DeepFake Detection using H.264 Motion Vectors

Authors: Peter Grönquist, Yufan Ren, Qingyi He, Alessio Verardo, Sabine Süsstrunk

Abstract: Video DeepFakes are fake media created with Deep Learning (DL) that manipulate a person's expression or identity. Most current DeepFake detection methods analyze each frame independently, ignoring inconsistencies and unnatural movements between frames. Some newer methods employ optical flow models to capture this temporal aspect, but they are computationally expensive. In contrast, we propose usin… ▽ More Video DeepFakes are fake media created with Deep Learning (DL) that manipulate a person's expression or identity. Most current DeepFake detection methods analyze each frame independently, ignoring inconsistencies and unnatural movements between frames. Some newer methods employ optical flow models to capture this temporal aspect, but they are computationally expensive. In contrast, we propose using the related but often ignored Motion Vectors (MVs) and Information Masks (IMs) from the H.264 video codec, to detect temporal inconsistencies in DeepFakes. Our experiments show that this approach is effective and has minimal computational costs, compared with per-frame RGB-only methods. This could lead to new, real-time temporally-aware DeepFake detection methods for video calls and streaming. △ Less

Submitted 22 February, 2024; v1 submitted 16 November, 2023; originally announced November 2023.

ACM Class: I.5.4; I.4.8; I.2.10; I.4.2

On computability of equilibrium states

Authors: Ilia Binder, Qiandu He, Zhiqiang Li, Yiwei Zhang

Abstract: Equilibrium states are natural dynamical analogs of Gibbs states in thermodynamic formalism. This paper is devoted to the study of their computability in the sense of Computable Analysis. We show that the unique equilibrium state associated to a pair of a computable, topologically exact, distance-expanding, open map $T\colon X\rightarrow X$ and a computable Hölder continuous potential… ▽ More Equilibrium states are natural dynamical analogs of Gibbs states in thermodynamic formalism. This paper is devoted to the study of their computability in the sense of Computable Analysis. We show that the unique equilibrium state associated to a pair of a computable, topologically exact, distance-expanding, open map $T\colon X\rightarrow X$ and a computable Hölder continuous potential $\varphi\colon X\rightarrow\mathbb{R}$ is always computable. Furthermore, the Hausdorff dimension of the Julia set and the equilibrium state for the geometric potential of a computable hyperbolic rational map are computable. On the other hand, we introduce a mechanism to establish non-uniqueness of equilibrium states. We also present some computable dynamical systems whose equilibrium states are all non-computable. △ Less

Submitted 14 January, 2024; v1 submitted 15 November, 2023; originally announced November 2023.

Comments: 41 pages. Reformatted, polished, typos corrected, Theorems D and E reformulated with Section 6 adjusted accordingly

MSC Class: Primary: 03D78; Secondary: 37D35; 37D20; 37F15

GazeForensics: DeepFake Detection via Gaze-guided Spatial Inconsistency Learning

Authors: Qinlin He, Chunlei Peng, Decheng Liu, Nannan Wang, Xinbo Gao

Abstract: DeepFake detection is pivotal in personal privacy and public safety. With the iterative advancement of DeepFake techniques, high-quality forged videos and images are becoming increasingly deceptive. Prior research has seen numerous attempts by scholars to incorporate biometric features into the field of DeepFake detection. However, traditional biometric-based approaches tend to segregate biometric… ▽ More DeepFake detection is pivotal in personal privacy and public safety. With the iterative advancement of DeepFake techniques, high-quality forged videos and images are becoming increasingly deceptive. Prior research has seen numerous attempts by scholars to incorporate biometric features into the field of DeepFake detection. However, traditional biometric-based approaches tend to segregate biometric features from general ones and freeze the biometric feature extractor. These approaches resulted in the exclusion of valuable general features, potentially leading to a performance decline and, consequently, a failure to fully exploit the potential of biometric information in assisting DeepFake detection. Moreover, insufficient attention has been dedicated to scrutinizing gaze authenticity within the realm of DeepFake detection in recent years. In this paper, we introduce GazeForensics, an innovative DeepFake detection method that utilizes gaze representation obtained from a 3D gaze estimation model to regularize the corresponding representation within our DeepFake detection model, while concurrently integrating general features to further enhance the performance of our model. Experiment results reveal that our proposed GazeForensics outperforms the current state-of-the-art methods. △ Less

Submitted 22 November, 2023; v1 submitted 12 November, 2023; originally announced November 2023.

Cloud-cloud collision in S235: triggered the formation of high-mass stars and young star clusters

Authors: En Chen, Yu Gao, Shiyu Zhang, Xuepeng Chen, Min Fang, Qianru He, Xuejian Jiang, Yan Sun, Xiaolong Wang, Hongjun Ma

Abstract: We present the analysis of cloud-cloud collision (CCC) process in the Galactic molecular complex S235. Our new CO observations performed with the PMO-13.7m telescope reveal two molecular clouds, namely the S235-Main and the S235-ABC, with $\sim$ 4 km s$^{-1}$ velocity separation. The bridge feature, the possible colliding interface and the complementary distribution of the two clouds are significa… ▽ More We present the analysis of cloud-cloud collision (CCC) process in the Galactic molecular complex S235. Our new CO observations performed with the PMO-13.7m telescope reveal two molecular clouds, namely the S235-Main and the S235-ABC, with $\sim$ 4 km s$^{-1}$ velocity separation. The bridge feature, the possible colliding interface and the complementary distribution of the two clouds are significant observational signatures of cloud-cloud collision in S235. The most direct evidence of cloud-cloud collision process in S235 is that the S235-Main (in a distance of 1547$^{+44}_{-43}$ pc) and S235-ABC (1567$^{+33}_{-39}$ pc) meet at almost the same position (within 1$σ$ error range) at a supersonic relative speed. We identified ten $^{13}$CO clumps from PMO-13.7m observations, 22 dust cores from the archival SCUBA-2 data, and 550 YSOs from NIR-MIR data. 63$\%$ of total YSOs are clustering in seven MST groups (M1$-$M7). The tight association between the YSO groups (M1 $\&$ M7) and the bridge feature suggests that the CCC process triggers star formation there. The collisional impact subregion (the South) shows $3\sim5$ times higher CFE and SFE (average value of 12.3$\%$ and 10.6$\%$, respectively) than the non-collisional impact subregion (2.4$\%$ and 2.6$\%$, respectively), suggesting that the CCC process may have enhanced the CFE and SFE of the clouds compared to those without collision influence. △ Less

Submitted 7 November, 2023; originally announced November 2023.

The Case for Controls: Identifying outbreak risk factors through case-control comparisons

Authors: Nina H. Fefferman, Michael J. Blum, Lydia Bourouiba, Nathaniel L. Gibson, Qiang He, Debra L. Miller, Monica Papes, Dana K. Pasquale, Connor Verheyen, Sadie J. Ryan

Abstract: Investigations of infectious disease outbreaks often focus on identifying place- and context-dependent factors responsible for emergence and spread, resulting in phenomenological narratives ill-suited to developing generalizable predictive and preventive measures. We contend that case-control hypothesis testing is a more powerful framework for epidemiological investigation. The approach, widely us… ▽ More Investigations of infectious disease outbreaks often focus on identifying place- and context-dependent factors responsible for emergence and spread, resulting in phenomenological narratives ill-suited to developing generalizable predictive and preventive measures. We contend that case-control hypothesis testing is a more powerful framework for epidemiological investigation. The approach, widely used in medical research, involves identifying counterfactuals, with case-control comparisons drawn to test hypotheses about the conditions that manifest outbreaks. Here we outline the merits of applying a case-control framework as epidemiological study design. We first describe a framework for iterative multidisciplinary interrogation to discover minimally sufficient sets of factors that can lead to disease outbreaks. We then lay out how case-control comparisons can respectively center on pathogen(s), factor(s), or landscape(s) with vignettes focusing on pathogen transmission. Finally, we consider how adopting case-control approaches can promote evidence-based decision making for responding to and preventing outbreaks. △ Less

Submitted 3 November, 2023; originally announced November 2023.

Two-dimensional double-kagome-lattice nitrogene: a direct band gap semiconductor with nontrivial corner state

Authors: Wenzhang Li, Qin He, Xiao-Ping Li, Da-Shuai Ma, Botao Fu

Abstract: Based on first-principles calculations, we predict that nitrogen atoms can assemble into a single-layer double kagome lattice (DKL), which possesses the characteristics of an intrinsic direct band gap semiconductor, boasting a substantial band gap of 3.460 eV. The DKL structure results in a flat valence band with high effective mass and a conduction band with small effective mass comes from Dirac… ▽ More Based on first-principles calculations, we predict that nitrogen atoms can assemble into a single-layer double kagome lattice (DKL), which possesses the characteristics of an intrinsic direct band gap semiconductor, boasting a substantial band gap of 3.460 eV. The DKL structure results in a flat valence band with high effective mass and a conduction band with small effective mass comes from Dirac electrons. These distinctive band edges lead to a significant disparity in carrier mobilities, with electron mobility being four orders of magnitude higher than that of holes. The presence of flat band in DKL-nitrogene can be further discerned through the enhanced optical absorption and correlated effects as exemplified by hole-induced ferromagnetism. Interestingly, DKL-nitrogene exhibits inherent second-order topological states, confirmed by a non-trivial second Stiefel-Whitney number and the presence of 1D floating edge states and 0D corner states within the bulk band gap. Additionally, the robust N-N bonds and the lattice's bending structure ensure thermodynamic stability and mechanical stiffness. These attributes make it exceptionally stable for potential applications in nano-devices. △ Less

Submitted 1 November, 2023; originally announced November 2023.

Comments: 9 pages, 4 figures

EdgeDis: Enabling Fast, Economical, and Reliable Data Dissemination for Mobile Edge Computing

Authors: Bo Li, Qiang He, Feifei Chen, Lingjuan Lyu, Athman Bouguettaya, Yun Yang

Abstract: Mobile edge computing (MEC) enables web data caching in close geographic proximity to end users. Popular data can be cached on edge servers located less than hundreds of meters away from end users. This ensures bounded latency guarantees for various latency-sensitive web applications. However, transmitting a large volume of data out of the cloud onto many geographically-distributed web servers ind… ▽ More Mobile edge computing (MEC) enables web data caching in close geographic proximity to end users. Popular data can be cached on edge servers located less than hundreds of meters away from end users. This ensures bounded latency guarantees for various latency-sensitive web applications. However, transmitting a large volume of data out of the cloud onto many geographically-distributed web servers individually can be expensive. In addition, web content dissemination may be interrupted by various intentional and accidental events in the volatile MEC environment, which undermines dissemination efficiency and subsequently incurs extra transmission costs. To tackle the above challenges, we present a novel scheme named EdgeDis that coordinates data dissemination by distributed consensus among those servers. We analyze EdgeDis's validity theoretically and evaluate its performance experimentally. Results demonstrate that compared with baseline and state-of-the-art schemes, EdgeDis: 1) is 5.97x - 7.52x faster; 2) reduces dissemination costs by 48.21% to 91.87%; and 3) reduces performance loss caused by dissemination failures by up to 97.30% in time and 96.35% in costs. △ Less

Submitted 31 October, 2023; originally announced November 2023.

Correlation-pattern-based Continuous-variable Entanglement Detection through Neural Networks

Authors: Xiaoting Gao, Mathieu Isoard, Fengxiao Sun, Carlos E. Lopetegui, Yu Xiang, Valentina Parigi, Qiongyi He, Mattia Walschaers

Abstract: Entanglement in continuous-variable non-Gaussian states provides irreplaceable advantages in many quantum information tasks. However, the sheer amount of information in such states grows exponentially and makes a full characterization impossible. Here, we develop a neural network that allows us to use correlation patterns to effectively detect continuous-variable entanglement through homodyne dete… ▽ More Entanglement in continuous-variable non-Gaussian states provides irreplaceable advantages in many quantum information tasks. However, the sheer amount of information in such states grows exponentially and makes a full characterization impossible. Here, we develop a neural network that allows us to use correlation patterns to effectively detect continuous-variable entanglement through homodyne detection. Using a recently defined stellar hierarchy to rank the states used for training, our algorithm works not only on any kind of Gaussian state but also on a whole class of experimentally achievable non-Gaussian states, including photon-subtracted states. With the same limited amount of data, our method provides higher accuracy than usual methods to detect entanglement based on maximum-likelihood tomography. Moreover, in order to visualize the effect of the neural network, we employ a dimension reduction algorithm on the patterns. This shows that a clear boundary appears between the entangled states and others after the neural network processing. In addition, these techniques allow us to compare different entanglement witnesses and understand their working. Our findings provide a new approach for experimental detection of continuous-variable quantum correlations without resorting to a full tomography of the state and confirm the exciting potential of neural networks in quantum information processing. △ Less

Submitted 31 October, 2023; originally announced October 2023.

Comments: 9 pages (incl. appendix), 6 figures, comments welcome!

Multi-Path Bound for DAG Tasks

Authors: Qingqiang He, Nan Guan, Shuai Zhao, Mingsong Lv

Abstract: This paper studies the response time bound of a DAG (directed acyclic graph) task. Recently, the idea of using multiple paths to bound the response time of a DAG task, instead of using a single longest path in previous results, was proposed and leads to the so-called multi-path bound. Multi-path bounds can greatly reduce the response time bound and significantly improve the schedulability of DAG t… ▽ More This paper studies the response time bound of a DAG (directed acyclic graph) task. Recently, the idea of using multiple paths to bound the response time of a DAG task, instead of using a single longest path in previous results, was proposed and leads to the so-called multi-path bound. Multi-path bounds can greatly reduce the response time bound and significantly improve the schedulability of DAG tasks. This paper derives a new multi-path bound and proposes an optimal algorithm to compute this bound. We further present a systematic analysis on the dominance and the sustainability of three existing multi-path bounds and the proposed multi-path bound. Our bound theoretically dominates and empirically outperforms all existing multi-path bounds. What's more, the proposed bound is the only multi-path bound that is proved to be self-sustainable. △ Less

Submitted 23 October, 2023; originally announced October 2023.

Assisted metrology and preparation of macroscopic superpositions with split spin-squeezed states

Authors: Jiajie Guo, Fengxiao Sun, Qiongyi He, Matteo Fadel

Abstract: We analyse the conditional states in which one part of a split spin-squeezed state is left, upon performing a collective spin measurement on the other part. For appropriate measurement directions and outcomes, we see the possibility of obtaining states with high quantum Fisher information, even reaching the Heisenberg limit. This allows us to propose a metrological protocol that can outperform sta… ▽ More We analyse the conditional states in which one part of a split spin-squeezed state is left, upon performing a collective spin measurement on the other part. For appropriate measurement directions and outcomes, we see the possibility of obtaining states with high quantum Fisher information, even reaching the Heisenberg limit. This allows us to propose a metrological protocol that can outperform standard approaches, for example in a situation where the number of particles in the probe is bounded. The robustness of this protocol is investigated by considering realistic forms of noise present in cold-atom experiments, such as particle number fluctuations and imperfect detection. Ultimately, we show how this measurement-based state preparation approach can allow for the conditional (\ie heralded) preparation of spin Schrödinger's cat states even when the initial state before splitting is only mildly squeezed. △ Less

Submitted 19 October, 2023; originally announced October 2023.

Journal ref: Phys. Rev. A 108, 053327 (2023)

Keep Various Trajectories: Promoting Exploration of Ensemble Policies in Continuous Control

Authors: Chao Li, Chen Gong, Qiang He, Xinwen Hou

Abstract: The combination of deep reinforcement learning (DRL) with ensemble methods has been proved to be highly effective in addressing complex sequential decision-making problems. This success can be primarily attributed to the utilization of multiple models, which enhances both the robustness of the policy and the accuracy of value function estimation. However, there has been limited analysis of the emp… ▽ More The combination of deep reinforcement learning (DRL) with ensemble methods has been proved to be highly effective in addressing complex sequential decision-making problems. This success can be primarily attributed to the utilization of multiple models, which enhances both the robustness of the policy and the accuracy of value function estimation. However, there has been limited analysis of the empirical success of current ensemble RL methods thus far. Our new analysis reveals that the sample efficiency of previous ensemble DRL algorithms may be limited by sub-policies that are not as diverse as they could be. Motivated by these findings, our study introduces a new ensemble RL algorithm, termed \textbf{T}rajectories-awar\textbf{E} \textbf{E}nsemble exploratio\textbf{N} (TEEN). The primary goal of TEEN is to maximize the expected return while promoting more diverse trajectories. Through extensive experiments, we demonstrate that TEEN not only enhances the sample diversity of the ensemble policy compared to using sub-policies alone but also improves the performance over ensemble RL algorithms. On average, TEEN outperforms the baseline ensemble DRL algorithms by 41\% in performance on the tested representative environments. △ Less

Submitted 17 October, 2023; originally announced October 2023.

New fractional-type weights and the boundedness of some operators

Authors: Xi Cen, Qianjun He

Abstract: The purpose of this study is twofold. On the one hand, we establish a new class of fractional-type multiple weights ${A_{\vec p( \cdot ),q( \cdot )}}$, which can be regarded as a variable exponents generalization of ${A_{\vec p,q}}$, a multilinear generalization of ${A_{p( \cdot ),q( \cdot )}}$, and a fractional-type generalization of ${A_{\vec p( \cdot )}}$. Moreover, it can be characterized by m… ▽ More The purpose of this study is twofold. On the one hand, we establish a new class of fractional-type multiple weights ${A_{\vec p( \cdot ),q( \cdot )}}$, which can be regarded as a variable exponents generalization of ${A_{\vec p,q}}$, a multilinear generalization of ${A_{p( \cdot ),q( \cdot )}}$, and a fractional-type generalization of ${A_{\vec p( \cdot )}}$. Moreover, it can be characterized by multilinear fractional-type operators ${\mathscr T}_α$. This generalizes the work by Moen (2009), Cruz-Uribe and Wang (2017), and Cruz-Uribe and Guzmán (2020), which is also called multilinear Hardy-Littlewood-Sobolev theorem on weighted variable exponents Lebesgue spaces. On the other hand, we also set up a new class of fractional-type variable matrix weights ${\mathbb{A}_{p( \cdot ),q( \cdot )}}$ that are characterized by fractional-type averaging operators ${\mathscr A}_{α,Q}$ and fractional Christ-Goldberg maximal operator ${{\mathcal M}_{α,W}}$. △ Less

Submitted 1 April, 2024; v1 submitted 16 October, 2023; originally announced October 2023.

Comments: 41 pages; 1 figure;

MSC Class: 42B20; 42B25; 42B35

How enlightened self-interest guided global vaccine sharing benefits all: a modelling study

Authors: Zhenyu Han, Qianyue Hao, Qiwei He, Katherine Budeski, Depeng Jin, Fengli Xu, Kun Tang

Abstract: Background: Despite the consensus that vaccines play an important role in combating the global spread of infectious diseases, vaccine inequity is still rampant with deep-seated mentality of self-priority. This study aims to evaluate the existence and possible outcomes of a more equitable global vaccine distribution and explore a concrete incentive mechanism that promotes vaccine equity. Methods: W… ▽ More Background: Despite the consensus that vaccines play an important role in combating the global spread of infectious diseases, vaccine inequity is still rampant with deep-seated mentality of self-priority. This study aims to evaluate the existence and possible outcomes of a more equitable global vaccine distribution and explore a concrete incentive mechanism that promotes vaccine equity. Methods: We design a metapopulation epidemiological model that simultaneously considers global vaccine distribution and human mobility, which is then calibrated by the number of infections and real-world vaccination records during COVID-19 pandemic from March 2020 to July 2021. We explore the possibility of the enlightened self-interest incentive mechanism, i.e., improving one's own epidemic outcomes by sharing vaccines with other countries, by evaluating the number of infections and deaths under various vaccine sharing strategies using the proposed model. To understand how these strategies affect the national interests, we distinguish the imported and local cases for further cost-benefit analyses that rationalize the enlightened self-interest incentive mechanism behind vaccine sharing. ... △ Less

Submitted 13 October, 2023; originally announced October 2023.

Comments: Accepted by Journal of Global Health

Multi-alpha Boson Gas state in Fusion Evaporation Reaction and Three-body Force

Authors: Taofeng Wang, Ziming Li, R. B. Wiringa, Minliang Liu, Jiansong Wang, Yanyun Yang, Qinghua He, Zhiyu Sun, Chengjian Lin, M. Assié, Y. Ayyad, D. Beaumel, Zhen Bai, Fangfang Duan, Zhihao Gao, Song Guo, Yue Hu, Wei Jiang, F. Kobayashi, Chengui Lu, Junbing Ma, Peng Ma, P. Napolitani, G. Verde, Jianguo Wang , et al. (11 additional authors not shown)

Abstract: The experimental evidence for the $α$ Boson gas state in the $^{11}$C+$^{12}$C$\rightarrow$$^{23}$Mg$^{\ast}$ fusion evaporation reaction is presented. By measuring the $α$ emission spectrum with multiplicity 2 and 3, we provide insight into the existence of a three-body force among $α$ particles. The observed spectrum exhibited distinct tails corresponding to $α$ particles emitted in pairs and tr… ▽ More The experimental evidence for the $α$ Boson gas state in the $^{11}$C+$^{12}$C$\rightarrow$$^{23}$Mg$^{\ast}$ fusion evaporation reaction is presented. By measuring the $α$ emission spectrum with multiplicity 2 and 3, we provide insight into the existence of a three-body force among $α$ particles. The observed spectrum exhibited distinct tails corresponding to $α$ particles emitted in pairs and triplets consistent well with the model-calculations of AV18-UX and chiral effective field theory of NV2-3-la*, indicating the formation of $α$ clusters with three-body force in the Boson gas state. △ Less

Submitted 6 October, 2023; originally announced October 2023.

Comments: 7 pages, 6 figures

Aspect of Clusters Correlation at Light Nuclei Excited State

Authors: Ziming Li, Jie Zhu, Taofeng Wang, Minliang Liu, Jiansong Wang, Yanyun Yang, Chengjian Lin, Zhiyu Sun, Qinghua He, M. Assié, Y. Ayyad, D. Beaumel, Zhen Bai, Fangfang Duan, Zhihao Gao, Song Guo, Yue Hu, Wei Jiang, F. Kobayashi, Chengui Lu, Junbing Ma, Peng Ma, P. Napolitani, G. Verde, Jianguo Wang , et al. (11 additional authors not shown)

Abstract: The correlation of $αα$ was probed via measuring the transverse momentum $p_{T}$ and width $δp_{T}$ of one $α$, for the first time, which represents the spatial and dynamical essentialities of the initial coupling state in $^{8}$Be nucleus. The weighted interaction vertex of 3$α$ reflected by the magnitudes of their relative momentums and relative emission angles proves the isosceles triangle conf… ▽ More The correlation of $αα$ was probed via measuring the transverse momentum $p_{T}$ and width $δp_{T}$ of one $α$, for the first time, which represents the spatial and dynamical essentialities of the initial coupling state in $^{8}$Be nucleus. The weighted interaction vertex of 3$α$ reflected by the magnitudes of their relative momentums and relative emission angles proves the isosceles triangle configuration for 3$α$ at the high excited energy analogous Hoyle states. △ Less

Submitted 6 October, 2023; originally announced October 2023.

Comments: 8 pages, 9 figures

Variation of Tensor Force due to Nuclear Medium Effect

Authors: Ziming Li, Jie Zhu, Taofeng Wang, Minliang Liu, Jiansong Wang, Yanyun Yang, Chengjian Lin, Zhiyu Sun, Qinghua He, M. Assié, Y. Ayyad, D. Beaumel, Zhen Bai, Fangfang Duan, Zhihao Gao, Song Guo, Yue Hu, Wei Jiang, F. Kobayashi, Chengui Lu, Junbing Ma, Peng Ma, P. Napolitani, G. Verde, Jianguo Wang , et al. (11 additional authors not shown)

Abstract: The enhancement of $J^π(T)$=3$^{+}$(0) state with isospin $T=0$ excited by the tensor force in the free $^{6}$Li nucleus has been observed, for the first time, relative to a shrinkable excitation in the $^{6}$Li cluster component inside its host nucleus. Comparatively, the excitation of $J^π(T)$=0$^{+}$(1) state with isospin $T=1$ for these two $^{6}$Li formations take on an approximately equal ex… ▽ More The enhancement of $J^π(T)$=3$^{+}$(0) state with isospin $T=0$ excited by the tensor force in the free $^{6}$Li nucleus has been observed, for the first time, relative to a shrinkable excitation in the $^{6}$Li cluster component inside its host nucleus. Comparatively, the excitation of $J^π(T)$=0$^{+}$(1) state with isospin $T=1$ for these two $^{6}$Li formations take on an approximately equal excitation strength. The mechanism of such tensor force effect was proposed due to the intensive nuclear medium role on isospin $T$=0 state. △ Less

Submitted 6 October, 2023; originally announced October 2023.

Comments: 6 pages, 4 figures

X-ray computed tomography reconstruction algorithm for refractive index gradient

Authors: Keliang Liao, Qili He, Panyun Li, Liang Luo, Peiping Zhu

Abstract: The aim of this research is to reconstruct the 3D X-ray refractive index gradient maps by the proposed vector Radon transform and its inverse, assuming that the small-angle deviation condition is met. Theoretical analyses show that the X-ray beam can be modeled as a streamline with continuous change of direction in a row when measured in one grating period, which allows the extraction of the refra… ▽ More The aim of this research is to reconstruct the 3D X-ray refractive index gradient maps by the proposed vector Radon transform and its inverse, assuming that the small-angle deviation condition is met. Theoretical analyses show that the X-ray beam can be modeled as a streamline with continuous change of direction in a row when measured in one grating period, which allows the extraction of the refraction angle signals. Experimental results show that all the 2D refraction signals of different directions can be acquired by a standard circular scanning procedure, which is typically used in the X-ray differential phase-contrast computed tomography. Furthermore, the 3D refractive index gradient maps that contain the directional density changes, can also be accurately reconstructed. △ Less

Submitted 4 September, 2023; originally announced September 2023.

Comments: accepted by IGTA2023

UGC: Unified GAN Compression for Efficient Image-to-Image Translation

Authors: Yuxi Ren, Jie Wu, Peng Zhang, Manlin Zhang, Xuefeng Xiao, Qian He, Rui Wang, Min Zheng, Xin Pan

Abstract: Recent years have witnessed the prevailing progress of Generative Adversarial Networks (GANs) in image-to-image translation. However, the success of these GAN models hinges on ponderous computational costs and labor-expensive training data. Current efficient GAN learning techniques often fall into two orthogonal aspects: i) model slimming via reduced calculation costs; ii)data/label-efficient lear… ▽ More Recent years have witnessed the prevailing progress of Generative Adversarial Networks (GANs) in image-to-image translation. However, the success of these GAN models hinges on ponderous computational costs and labor-expensive training data. Current efficient GAN learning techniques often fall into two orthogonal aspects: i) model slimming via reduced calculation costs; ii)data/label-efficient learning with fewer training data/labels. To combine the best of both worlds, we propose a new learning paradigm, Unified GAN Compression (UGC), with a unified optimization objective to seamlessly prompt the synergy of model-efficient and label-efficient learning. UGC sets up semi-supervised-driven network architecture search and adaptive online semi-supervised distillation stages sequentially, which formulates a heterogeneous mutual learning scheme to obtain an architecture-flexible, label-efficient, and performance-excellent model. △ Less

Submitted 17 September, 2023; originally announced September 2023.

Can Large Language Models Understand Real-World Complex Instructions?

Authors: Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Lida Chen, Xintao Wang, Yuncheng Huang, Haoning Ye, Zihan Li, Shisong Chen, Yikai Zhang, Zhouhong Gu, Jiaqing Liang, Yanghua Xiao

Abstract: Large language models (LLMs) can understand human instructions, showing their potential for pragmatic applications beyond traditional NLP tasks. However, they still struggle with complex instructions, which can be either complex task descriptions that require multiple tasks and constraints, or complex input that contains long context, noise, heterogeneous information and multi-turn format. Due to… ▽ More Large language models (LLMs) can understand human instructions, showing their potential for pragmatic applications beyond traditional NLP tasks. However, they still struggle with complex instructions, which can be either complex task descriptions that require multiple tasks and constraints, or complex input that contains long context, noise, heterogeneous information and multi-turn format. Due to these features, LLMs often ignore semantic constraints from task descriptions, generate incorrect formats, violate length or sample count constraints, and be unfaithful to the input text. Existing benchmarks are insufficient to assess LLMs' ability to understand complex instructions, as they are close-ended and simple. To bridge this gap, we propose CELLO, a benchmark for evaluating LLMs' ability to follow complex instructions systematically. We design eight features for complex instructions and construct a comprehensive evaluation dataset from real-world scenarios. We also establish four criteria and develop corresponding metrics, as current ones are inadequate, biased or too strict and coarse-grained. We compare the performance of representative Chinese-oriented and English-oriented models in following complex instructions through extensive experiments. Resources of CELLO are publicly available at https://github.com/Abbey4799/CELLO. △ Less

Submitted 8 January, 2024; v1 submitted 17 September, 2023; originally announced September 2023.

SpeechMirror: A Multimodal Visual Analytics System for Personalized Reflection of Online Public Speaking Effectiveness

Authors: Zeyuan Huang, Qiang He, Kevin Maher, Xiaoming Deng, Yu-Kun Lai, Cuixia Ma, Sheng-feng Qin, Yong-Jin Liu, Hongan Wang

Abstract: As communications are increasingly taking place virtually, the ability to present well online is becoming an indispensable skill. Online speakers are facing unique challenges in engaging with remote audiences. However, there has been a lack of evidence-based analytical systems for people to comprehensively evaluate online speeches and further discover possibilities for improvement. This paper intr… ▽ More As communications are increasingly taking place virtually, the ability to present well online is becoming an indispensable skill. Online speakers are facing unique challenges in engaging with remote audiences. However, there has been a lack of evidence-based analytical systems for people to comprehensively evaluate online speeches and further discover possibilities for improvement. This paper introduces SpeechMirror, a visual analytics system facilitating reflection on a speech based on insights from a collection of online speeches. The system estimates the impact of different speech techniques on effectiveness and applies them to a speech to give users awareness of the performance of speech techniques. A similarity recommendation approach based on speech factors or script content supports guided exploration to expand knowledge of presentation evidence and accelerate the discovery of speech delivery possibilities. SpeechMirror provides intuitive visualizations and interactions for users to understand speech factors. Among them, SpeechTwin, a novel multimodal visual summary of speech, supports rapid understanding of critical speech factors and comparison of different speech samples, and SpeechPlayer augments the speech video by integrating visualization of the speaker's body language with interaction, for focused analysis. The system utilizes visualizations suited to the distinct nature of different speech factors for user comprehension. The proposed system and visualization techniques were evaluated with domain experts and amateurs, demonstrating usability for users with low visualization literacy and its efficacy in assisting users to develop insights for potential improvement. △ Less

Submitted 10 September, 2023; originally announced September 2023.

Comments: Main paper (11 pages, 6 figures) and Supplemental document (11 pages, 11 figures). Accepted by VIS 2023