Showing 1–50 of 900 results for author: Zhu, M

Deep extragalactic HI survey of the COSMOS field with FAST

Authors: Hengxing Pan, Matt J. Jarvis, Ming Zhu, Yin-Zhe Ma, Mario G. Santos, Anastasia A. Ponomareva, Ian Heywood, Yingjie Jing, Chen Xu, Ziming Liu, Yogesh Chandola, Yipeng Jing

Abstract: We present a deep HI survey at L-band conducted with the Five-hundred-meter Aperture Spherical radio Telescope (FAST) over the COSMOS field. This survey is strategically designed to overlap with the MIGHTEE COSMOS field, aiming to combine the sensitivity of the FAST and high-resolution of the MeerKAT. We observed the field with FAST for 11 hours covering $\sim$2 square degrees, and reduced the raw… ▽ More We present a deep HI survey at L-band conducted with the Five-hundred-meter Aperture Spherical radio Telescope (FAST) over the COSMOS field. This survey is strategically designed to overlap with the MIGHTEE COSMOS field, aiming to combine the sensitivity of the FAST and high-resolution of the MeerKAT. We observed the field with FAST for 11 hours covering $\sim$2 square degrees, and reduced the raw data to HI spectral cubes over the frequency range 1310-1420 MHz. The FAST-HI data reach a median 3$σ$ column density of $N_{\rm HI}\sim2\times10^{17}$ cm$^{-2}$ over a 5 km s$^{-1}$ channel width, allowing for studies of the distribution of HI gas in various environments, such as in galaxies, the Circum-Galactic Medium (CGM) and Intergalactic Medium (IGM). We visually searched the spectral cubes for HI sources, and found a total of 80 HI detections, of which 56 have been cross-matched with the MIGHTEE-HI catalogue. With the cross-matched sources, we compare their HI masses and find that the total HI mass fraction in the IGM and CGM surrounding the galaxy pairs is statistically higher than the HI fraction surrounding the isolated galaxies by a difference of 13$\pm$4%, indicating that the CGM and IGM associated with interacting systems are richer in neutral hydrogen compared to those around isolated galaxies in the local Universe. We also describe several FAST-MeerKAT synergy projects, highlighting the full potential of exploiting both single-dish and interferometric observations to study the distribution and evolution of the diffuse HI gas. △ Less

Submitted 29 August, 2024; originally announced August 2024.

Comments: 13 pages, 14 figures; Accepted for publication in MNRAS

Protograph-Based Batched Network Codes

Authors: Mingyang Zhu, Ming Jiang, Chunming Zhao

Abstract: Batched network codes (BNCs) are a low-complexity solution for communication through networks with packet loss. Although their belief propagation (BP) performance is proved to approach capacity in the asymptotic regime, there is no evidence indicating that their BP performance is as good as expected in the finite-length regime. In this paper, we propose a protograph-based construction for BNCs, re… ▽ More Batched network codes (BNCs) are a low-complexity solution for communication through networks with packet loss. Although their belief propagation (BP) performance is proved to approach capacity in the asymptotic regime, there is no evidence indicating that their BP performance is as good as expected in the finite-length regime. In this paper, we propose a protograph-based construction for BNCs, referred to as protograph-based BNCs (P-BNCs), which significantly differs from existing BNCs in three aspects: 1) Unlike traditional constructions where the degree of variable nodes is random, P-BNCs have a highly structured Tanner graph with specified degree distributions for both variable nodes and check nodes. 2) Traditional BNCs use a fixed degree distribution to generate all batches, making their performance highly sensitive to channel conditions, but P-BNCs achieve good performance under varying channel conditions due to their rate-compatible structures. 3) The construction of PBNCs takes into account joint BP decoding with a sparse precode, whereas traditional constructions typically do not consider a precode, or assume the presence of a precode that can recover a certain fraction of erasures. Thanks to these three improvements, P-BNCs not only have higher achievable rates under varying channel conditions, but more importantly, their finite-length BP performance is significantly improved. △ Less

Submitted 29 August, 2024; originally announced August 2024.

Comments: submitted to IEEE for possible publication

Foodfusion: A Novel Approach for Food Image Composition via Diffusion Models

Authors: Chaohua Shi, Xuan Wang, Si Shi, Xule Wang, Mingrui Zhu, Nannan Wang, Xinbo Gao

Abstract: Food image composition requires the use of existing dish images and background images to synthesize a natural new image, while diffusion models have made significant advancements in image generation, enabling the construction of end-to-end architectures that yield promising results. However, existing diffusion models face challenges in processing and fusing information from multiple images and lac… ▽ More Food image composition requires the use of existing dish images and background images to synthesize a natural new image, while diffusion models have made significant advancements in image generation, enabling the construction of end-to-end architectures that yield promising results. However, existing diffusion models face challenges in processing and fusing information from multiple images and lack access to high-quality publicly available datasets, which prevents the application of diffusion models in food image composition. In this paper, we introduce a large-scale, high-quality food image composite dataset, FC22k, which comprises 22,000 foreground, background, and ground truth ternary image pairs. Additionally, we propose a novel food image composition method, Foodfusion, which leverages the capabilities of the pre-trained diffusion models and incorporates a Fusion Module for processing and integrating foreground and background information. This fused information aligns the foreground features with the background structure by merging the global structural information at the cross-attention layer of the denoising UNet. To further enhance the content and structure of the background, we also integrate a Content-Structure Control Module. Extensive experiments demonstrate the effectiveness and scalability of our proposed method. △ Less

Submitted 26 August, 2024; originally announced August 2024.

Comments: 14 pages

IntOPE: Off-Policy Evaluation in the Presence of Interference

Authors: Yuqi Bai, Ziyu Zhao, Minqin Zhu, Kun Kuang

Abstract: Off-Policy Evaluation (OPE) is employed to assess the potential impact of a hypothetical policy using logged contextual bandit feedback, which is crucial in areas such as personalized medicine and recommender systems, where online interactions are associated with significant risks and costs. Traditionally, OPE methods rely on the Stable Unit Treatment Value Assumption (SUTVA), which assumes that t… ▽ More Off-Policy Evaluation (OPE) is employed to assess the potential impact of a hypothetical policy using logged contextual bandit feedback, which is crucial in areas such as personalized medicine and recommender systems, where online interactions are associated with significant risks and costs. Traditionally, OPE methods rely on the Stable Unit Treatment Value Assumption (SUTVA), which assumes that the reward for any given individual is unaffected by the actions of others. However, this assumption often fails in real-world scenarios due to the presence of interference, where an individual's reward is affected not just by their own actions but also by the actions of their peers. This realization reveals significant limitations of existing OPE methods in real-world applications. To address this limitation, we propose IntIPW, an IPW-style estimator that extends the Inverse Probability Weighting (IPW) framework by integrating marginalized importance weights to account for both individual actions and the influence of adjacent entities. Extensive experiments are conducted on both synthetic and real-world data to demonstrate the effectiveness of the proposed IntIPW method. △ Less

Submitted 24 August, 2024; originally announced August 2024.

Perturbation on Feature Coalition: Towards Interpretable Deep Neural Networks

Authors: Xuran Hu, Mingzhe Zhu, Zhenpeng Feng, Miloš Daković, Ljubiša Stanković

Abstract: The inherent "black box" nature of deep neural networks (DNNs) compromises their transparency and reliability. Recently, explainable AI (XAI) has garnered increasing attention from researchers. Several perturbation-based interpretations have emerged. However, these methods often fail to adequately consider feature dependencies. To solve this problem, we introduce a perturbation-based interpretatio… ▽ More The inherent "black box" nature of deep neural networks (DNNs) compromises their transparency and reliability. Recently, explainable AI (XAI) has garnered increasing attention from researchers. Several perturbation-based interpretations have emerged. However, these methods often fail to adequately consider feature dependencies. To solve this problem, we introduce a perturbation-based interpretation guided by feature coalitions, which leverages deep information of network to extract correlated features. Then, we proposed a carefully-designed consistency loss to guide network interpretation. Both quantitative and qualitative experiments are conducted to validate the effectiveness of our proposed method. Code is available at github.com/Teriri1999/Perturebation-on-Feature-Coalition. △ Less

Submitted 23 August, 2024; originally announced August 2024.

Comments: 4 pages, 4 figures, 2 tables

Integrals of motion in conformal field theory with W-symmetry and the ODE/IM correspondence

Authors: Katsushi Ito, Mingshuo Zhu

Abstract: We study the ODE/IM correspondence between two-dimensional $WA_{r}$/$WD_{r}$-type conformal field theories and the higher-order ordinary differential equations (ODEs) obtained from the affine Toda field theories associated with $A_r^{(1)}/D_r^{(1)}$-type affine Lie algebras. We calculate the period integrals of the WKB solution to the ODE along the Pochhammer contour, where the WKB expansions corr… ▽ More We study the ODE/IM correspondence between two-dimensional $WA_{r}$/$WD_{r}$-type conformal field theories and the higher-order ordinary differential equations (ODEs) obtained from the affine Toda field theories associated with $A_r^{(1)}/D_r^{(1)}$-type affine Lie algebras. We calculate the period integrals of the WKB solution to the ODE along the Pochhammer contour, where the WKB expansions correspond to the classical conserved currents of the Drinfeld-Sokolov integrable hierarchies. We also compute the integrals of motion for $WA_{r}$($WD_{r}$) algebras on a cylinder. Their eigenvalues on the vacuum state are confirmed to agree with the period integrals up to the sixth order. These results generalize the ODE/IM correspondence to higher-order ODEs and can be used to predict higher-order integrals of motion. △ Less

Submitted 23 August, 2024; originally announced August 2024.

Comments: 1+47 pages, 1 figure

Report number: TIT/HEP-703

Personality Alignment of Large Language Models

Authors: Minjun Zhu, Linyi Yang, Yue Zhang

Abstract: Current methods for aligning large language models (LLMs) typically aim to reflect general human values and behaviors, but they often fail to capture the unique characteristics and preferences of individual users. To address this gap, we introduce the concept of Personality Alignment. This approach tailors LLMs' responses and decisions to match the specific preferences of individual users or close… ▽ More Current methods for aligning large language models (LLMs) typically aim to reflect general human values and behaviors, but they often fail to capture the unique characteristics and preferences of individual users. To address this gap, we introduce the concept of Personality Alignment. This approach tailors LLMs' responses and decisions to match the specific preferences of individual users or closely related groups. Inspired by psychometrics, we created the Personality Alignment with Personality Inventories (PAPI) dataset, which includes data from 300,000 real subjects, each providing behavioral preferences based on the Big Five Personality Factors. This dataset allows us to quantitatively evaluate the extent to which LLMs can align with each subject's behavioral patterns. Recognizing the challenges of personality alignments: such as limited personal data, diverse preferences, and scalability requirements: we developed an activation intervention optimization method. This method enhances LLMs' ability to efficiently align with individual behavioral preferences using minimal data and computational resources. Remarkably, our method, PAS, achieves superior performance while requiring only 1/5 of the optimization time compared to DPO, offering practical value for personality alignment. Our work paves the way for future AI systems to make decisions and reason in truly personality ways, enhancing the relevance and meaning of AI interactions for each user and advancing human-centered artificial intelligence.The code has released in \url{https://github.com/zhu-minjun/PAlign}. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Bridging experiment and theory of relaxor ferroelectrics at the atomic scale with multislice electron ptychography

Authors: Menglin Zhu, Michael Xu, Yubo Qi, Colin Gilgenbach, Jieun Kim, Jiahao Zhang, Bridget R. Denzer, Lane W. Martin, Andrew M. Rappe, James M. LeBeau

Abstract: Introducing structural and/or chemical heterogeneity into otherwise ordered crystals can dramatically alter material properties. Lead-based relaxor ferroelectrics are a prototypical example, with decades of investigation having connected chemical and structural heterogeneity to their unique properties. While theory has pointed to the formation of an ensemble of ``slush''-like polar domains, the la… ▽ More Introducing structural and/or chemical heterogeneity into otherwise ordered crystals can dramatically alter material properties. Lead-based relaxor ferroelectrics are a prototypical example, with decades of investigation having connected chemical and structural heterogeneity to their unique properties. While theory has pointed to the formation of an ensemble of ``slush''-like polar domains, the lack of direct, spatially resolved volumetric data comparable to simulations presents a significant challenge in measuring the spatial distribution and correlation of local chemistry and structure with the physics underlying relaxor behavior. Here, we address this challenge through three-dimensional volumetric characterization of the prototypical relaxor ferroelectric \ce{0.68Pb(Mg$_{1/3}$Nb$_{2/3}$)O3-0.32PbTiO$_3$} using multislice electron ptychography. Direct comparison with molecular dynamics simulations reveals the intimate relationship between the polar structure and unit-cell level charge imbalance induced by chemical disorder. Further, polar nanodomains are maintained through local correlations arising from residual short-range chemical order. Acting in concert with the chemical heterogeneities, it is also shown that compressive strain enhances out-of-plane correlations and ferroelectric-like order without affecting the in-plane relaxor-like structure. Broadly, these findings provide a pathway to enable detailed atomic scale understanding for hierarchical control of polar domains in relaxor ferroelectric materials and devices, and also present significant opportunities to tackle other heterogeneous systems using complementary theoretical and experimental studies. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Optical refractive index and spacetime geometry

Authors: Hui Xiong, Mengcheng Zhu

Abstract: Classical mechanics and geometrical optics are deeply connected with each other. In this work, we generalize the analogy between these two disciplines to relativistic conditions. Using this analogy, we are able to make light follow the orbits of massive or massless particles in gravitational field, by designing a particular optical medium with a prescribed refractive index profile according to spa… ▽ More Classical mechanics and geometrical optics are deeply connected with each other. In this work, we generalize the analogy between these two disciplines to relativistic conditions. Using this analogy, we are able to make light follow the orbits of massive or massless particles in gravitational field, by designing a particular optical medium with a prescribed refractive index profile according to spacetime geometry. Furthermore, we build the dictionary that connects optical refractive index and spacetime metric, thus extending F = ma optics to the relativistic limit. The results shed light on the equivalence between the dielectric properties of optical media and the geometric structure of spacetime. △ Less

Submitted 21 August, 2024; v1 submitted 20 August, 2024; originally announced August 2024.

Comments: 30 pages, 8 figures

Existence of normalized solutions to Choquard equation with general mixed nonlinearities

Authors: Meiling Zhu, Xinfu Li

Abstract: We study the existence of normalized solutions to the following Choquard equation with $F$ being a Berestycki-Lions type function \begin{equation*} \begin{cases} -Δu+λu=(I_α\ast F(u))f(u),\quad \text{in}\ \mathbb{R}^N, \\ \int_{\mathbb{R}^N}|u|^2dx=ρ^2, \end{cases} \end{equation*} where $N\geq 3$, $ρ>0$ is assigned, $α\in (0,N)$, $I_α$ is the Riesz potential, and $λ\in \mathbb{R}$ is an unknown pa… ▽ More We study the existence of normalized solutions to the following Choquard equation with $F$ being a Berestycki-Lions type function \begin{equation*} \begin{cases} -Δu+λu=(I_α\ast F(u))f(u),\quad \text{in}\ \mathbb{R}^N, \\ \int_{\mathbb{R}^N}|u|^2dx=ρ^2, \end{cases} \end{equation*} where $N\geq 3$, $ρ>0$ is assigned, $α\in (0,N)$, $I_α$ is the Riesz potential, and $λ\in \mathbb{R}$ is an unknown parameter that appears as a Lagrange multiplier. Here, the general nonlinearity $F$ contains the $L^2$-subcritical and $L^2$-supercritical mixed case, the Hardy-Littlewood-Sobolev lower critical and upper critical cases. △ Less

Submitted 19 August, 2024; originally announced August 2024.

MePT: Multi-Representation Guided Prompt Tuning for Vision-Language Model

Authors: Xinyang Wang, Yi Yang, Minfeng Zhu, Kecheng Zheng, Shi Liu, Wei Chen

Abstract: Recent advancements in pre-trained Vision-Language Models (VLMs) have highlighted the significant potential of prompt tuning for adapting these models to a wide range of downstream tasks. However, existing prompt tuning methods typically map an image to a single representation, limiting the model's ability to capture the diverse ways an image can be described. To address this limitation, we invest… ▽ More Recent advancements in pre-trained Vision-Language Models (VLMs) have highlighted the significant potential of prompt tuning for adapting these models to a wide range of downstream tasks. However, existing prompt tuning methods typically map an image to a single representation, limiting the model's ability to capture the diverse ways an image can be described. To address this limitation, we investigate the impact of visual prompts on the model's generalization capability and introduce a novel method termed Multi-Representation Guided Prompt Tuning (MePT). Specifically, MePT employs a three-branch framework that focuses on diverse salient regions, uncovering the inherent knowledge within images which is crucial for robust generalization. Further, we employ efficient self-ensemble techniques to integrate these versatile image representations, allowing MePT to learn all conditional, marginal, and fine-grained distributions effectively. We validate the effectiveness of MePT through extensive experiments, demonstrating significant improvements on both base-to-novel class prediction and domain generalization tasks. △ Less

Submitted 19 August, 2024; originally announced August 2024.

AIE: Auction Information Enhanced Framework for CTR Prediction in Online Advertising

Authors: Yang Yang, Bo Chen, Chenxu Zhu, Menghui Zhu, Xinyi Dai, Huifeng Guo, Muyu Zhang, Zhenhua Dong, Ruiming Tang

Abstract: Click-Through Rate (CTR) prediction is a fundamental technique for online advertising recommendation and the complex online competitive auction process also brings many difficulties to CTR optimization. Recent studies have shown that introducing posterior auction information contributes to the performance of CTR prediction. However, existing work doesn't fully capitalize on the benefits of auction… ▽ More Click-Through Rate (CTR) prediction is a fundamental technique for online advertising recommendation and the complex online competitive auction process also brings many difficulties to CTR optimization. Recent studies have shown that introducing posterior auction information contributes to the performance of CTR prediction. However, existing work doesn't fully capitalize on the benefits of auction information and overlooks the data bias brought by the auction, leading to biased and suboptimal results. To address these limitations, we propose Auction Information Enhanced Framework (AIE) for CTR prediction in online advertising, which delves into the problem of insufficient utilization of auction signals and first reveals the auction bias. Specifically, AIE introduces two pluggable modules, namely Adaptive Market-price Auxiliary Module (AM2) and Bid Calibration Module (BCM), which work collaboratively to excavate the posterior auction signals better and enhance the performance of CTR prediction. Furthermore, the two proposed modules are lightweight, model-agnostic, and friendly to inference latency. Extensive experiments are conducted on a public dataset and an industrial dataset to demonstrate the effectiveness and compatibility of AIE. Besides, a one-month online A/B test in a large-scale advertising platform shows that AIE improves the base model by 5.76% and 2.44% in terms of eCPM and CTR, respectively. △ Less

Submitted 14 August, 2024; originally announced August 2024.

One Step Diffusion-based Super-Resolution with Time-Aware Distillation

Authors: Xiao He, Huaao Tang, Zhijun Tu, Junchao Zhang, Kun Cheng, Hanting Chen, Yong Guo, Mingrui Zhu, Nannan Wang, Xinbo Gao, Jie Hu

Abstract: Diffusion-based image super-resolution (SR) methods have shown promise in reconstructing high-resolution images with fine details from low-resolution counterparts. However, these approaches typically require tens or even hundreds of iterative samplings, resulting in significant latency. Recently, techniques have been devised to enhance the sampling efficiency of diffusion-based SR models via knowl… ▽ More Diffusion-based image super-resolution (SR) methods have shown promise in reconstructing high-resolution images with fine details from low-resolution counterparts. However, these approaches typically require tens or even hundreds of iterative samplings, resulting in significant latency. Recently, techniques have been devised to enhance the sampling efficiency of diffusion-based SR models via knowledge distillation. Nonetheless, when aligning the knowledge of student and teacher models, these solutions either solely rely on pixel-level loss constraints or neglect the fact that diffusion models prioritize varying levels of information at different time steps. To accomplish effective and efficient image super-resolution, we propose a time-aware diffusion distillation method, named TAD-SR. Specifically, we introduce a novel score distillation strategy to align the data distribution between the outputs of the student and teacher models after minor noise perturbation. This distillation strategy enables the student network to concentrate more on the high-frequency details. Furthermore, to mitigate performance limitations stemming from distillation, we integrate a latent adversarial loss and devise a time-aware discriminator that leverages diffusion priors to effectively distinguish between real images and generated images. Extensive experiments conducted on synthetic and real-world datasets demonstrate that the proposed method achieves comparable or even superior performance compared to both previous state-of-the-art (SOTA) methods and the teacher model in just one sampling step. Codes are available at https://github.com/LearningHx/TAD-SR. △ Less

Submitted 14 August, 2024; originally announced August 2024.

Comments: 18 pages

Interpretating Pulsar Timing Array data of Gravitational Waves with Ekpyrosis-Bouncing Cosmology

Authors: Taotao Qiu, Mian Zhu

Abstract: Recent pulsar timing array (PTA) experiments have reported strong evidence of the stochastic gravitational wave background (SGWB). If interpreted as primordial Gravitational Waves (pGWs), the signal favors a strongly blue-tilted spectrum. On the other hand, the Ekpyrosis-bouncing cosmology with a strongly blue-tilted GW spectrum, i.e., $n_T \simeq 2$, offers a potential explanation for the observe… ▽ More Recent pulsar timing array (PTA) experiments have reported strong evidence of the stochastic gravitational wave background (SGWB). If interpreted as primordial Gravitational Waves (pGWs), the signal favors a strongly blue-tilted spectrum. On the other hand, the Ekpyrosis-bouncing cosmology with a strongly blue-tilted GW spectrum, i.e., $n_T \simeq 2$, offers a potential explanation for the observed SGWB signal. In this paper, we construct a concrete Ekpyrosis-bouncing model, and show its capacity to intepret the PTA result without pathologies. Both tensor and scalar perturbations are analysed with constraints from the current observations. △ Less

Submitted 22 August, 2024; v1 submitted 12 August, 2024; originally announced August 2024.

Comments: 24 pages, 4 figures and 1 table

A Decoding Acceleration Framework for Industrial Deployable LLM-based Recommender Systems

Authors: Yunjia Xi, Hangyu Wang, Bo Chen, Jianghao Lin, Menghui Zhu, Weiwen Liu, Ruiming Tang, Weinan Zhang, Yong Yu

Abstract: Recently, increasing attention has been paid to LLM-based recommender systems, but their deployment is still under exploration in the industry. Most deployments utilize LLMs as feature enhancers, generating augmentation knowledge in the offline stage. However, in recommendation scenarios, involving numerous users and items, even offline generation with LLMs consumes considerable time and resources… ▽ More Recently, increasing attention has been paid to LLM-based recommender systems, but their deployment is still under exploration in the industry. Most deployments utilize LLMs as feature enhancers, generating augmentation knowledge in the offline stage. However, in recommendation scenarios, involving numerous users and items, even offline generation with LLMs consumes considerable time and resources. This generation inefficiency stems from the autoregressive nature of LLMs, and a promising direction for acceleration is speculative decoding, a Draft-then-Verify paradigm that increases the number of generated tokens per decoding step. In this paper, we first identify that recommendation knowledge generation is suitable for retrieval-based speculative decoding. Then, we discern two characteristics: (1) extensive items and users in RSs bring retrieval inefficiency, and (2) RSs exhibit high diversity tolerance for text generated by LLMs. Based on the above insights, we propose a Decoding Acceleration Framework for LLM-based Recommendation (dubbed DARE), with Customized Retrieval Pool to improve retrieval efficiency and Relaxed Verification to increase the acceptance rate of draft tokens, respectively. Extensive experiments demonstrate that DARE achieves a 3-5x speedup and is compatible with various frameworks and backbone LLMs. DARE has also been deployed to online advertising scenarios within a large-scale commercial environment, achieving a 3.45x speedup while maintaining the downstream performance. △ Less

Submitted 10 August, 2024; originally announced August 2024.

CMOS-Compatible Ultrathin Superconducting NbN Thin Films Deposited by Reactive Ion Sputtering on 300 mm Si Wafer

Authors: Zihao Yang, Xiucheng Wei, Pinku Roy, Di Zhang, Ping Lu, Samyak Dhole, Haiyan Wang, Nicholas Cucciniello, Nag Patibandla, Zhebo Chen, Hao Zeng, Quanxi Jia, Mingwei Zhu

Abstract: We report a milestone in achieving large-scale, ultrathin (~5 nm) superconducting NbN thin films on 300 mm Si wafers using a high-volume manufacturing (HVM) industrial physical vapor deposition (PVD) system. The NbN thin films possess remarkable structural uniformity and consistently high superconducting quality across the entire 300 mm Si wafer, by incorporating an AlN buffer layer. High-resoluti… ▽ More We report a milestone in achieving large-scale, ultrathin (~5 nm) superconducting NbN thin films on 300 mm Si wafers using a high-volume manufacturing (HVM) industrial physical vapor deposition (PVD) system. The NbN thin films possess remarkable structural uniformity and consistently high superconducting quality across the entire 300 mm Si wafer, by incorporating an AlN buffer layer. High-resolution X-ray diffraction and transmission electron microscopy analyses unveiled enhanced crystallinity of (111)-oriented δ-phase NbN with the AlN buffer layer. Notably, NbN films deposited on AlN-buffered Si substrates exhibited a significantly elevated superconducting critical temperature (~2 K higher for the 10 nm NbN) and a higher upper critical magnetic field or Hc2 (34.06 T boost in Hc2 for the 50 nm NbN) in comparison with those without AlN. These findings present a promising pathway for the integration of quantum-grade superconducting NbN films with the existing 300 mm CMOS Si platform for quantum information applications. △ Less

Submitted 10 August, 2024; originally announced August 2024.

Journal ref: Materials 2023, 16, 7468

Exploring the origin of cold gas and star formation in a rare population of strongly bulge-dominated early-type Galaxies

Authors: Fujia Li, Enci Wang, Ming Zhu, Yingjie Peng, Jing Wang, Chuanpeng Zhang, Zesen Lin, Yu Rong, Hongxin Zhang, Xu Kong

Abstract: We analyze the properties of a rare population, the strongly bulge-dominated early-type galaxies (referred to as sBDEs) with significant HI gas, using the databases from the FAST All Sky HI survey (FASHI) and the Arecibo Legacy Fast ALFA (ALFALFA) survey. We select the sBDEs from the Sloan Digital Sky Survey (SDSS) and cross-match with the FASHI-ALFALFA combined HI sample, resulting in 104 HI-rich… ▽ More We analyze the properties of a rare population, the strongly bulge-dominated early-type galaxies (referred to as sBDEs) with significant HI gas, using the databases from the FAST All Sky HI survey (FASHI) and the Arecibo Legacy Fast ALFA (ALFALFA) survey. We select the sBDEs from the Sloan Digital Sky Survey (SDSS) and cross-match with the FASHI-ALFALFA combined HI sample, resulting in 104 HI-rich sBDEs. These sBDEs tend to have extremely high HI reservoirs, which is rare in previous studies such as ATLAS$^{3D}$. 70% of the selected sBDEs are classified as quiescent galaxies, even though they have a large HI reservoir. We study the properties of these sBDEs from five main aspects: stellar population, gas-phase metallicity, stacked HI spectra, environment, and spatially resolved MaNGA data. The majority of HI-rich sBDEs appear to show lower gas-phase metallicity and are located in significantly lower-density environments, suggesting an external origin for their HI gas. We find that star-forming sBDEs exhibit statistically higher star formation efficiency and slightly older stellar populations compared to normal star-forming galaxies, suggesting a recent star formation on Gyr-timescale. They also show narrower and more concentrated HI profiles compared to control star-forming galaxies, which may explain their higher star formation efficiency. △ Less

Submitted 8 August, 2024; originally announced August 2024.

Comments: 18 pages, 14 figures, 1 table. Accepted for publication in ApJ

EcoFollower: An Environment-Friendly Car Following Model Considering Fuel Consumption

Authors: Hui Zhong, Xianda Chen, PakHin Tiu, Hongliang Lu, Meixin Zhu

Abstract: To alleviate energy shortages and environmental impacts caused by transportation, this study introduces EcoFollower, a novel eco-car-following model developed using reinforcement learning (RL) to optimize fuel consumption in car-following scenarios. Employing the NGSIM datasets, the performance of EcoFollower was assessed in comparison with the well-established Intelligent Driver Model (IDM). The… ▽ More To alleviate energy shortages and environmental impacts caused by transportation, this study introduces EcoFollower, a novel eco-car-following model developed using reinforcement learning (RL) to optimize fuel consumption in car-following scenarios. Employing the NGSIM datasets, the performance of EcoFollower was assessed in comparison with the well-established Intelligent Driver Model (IDM). The findings demonstrate that EcoFollower excels in simulating realistic driving behaviors, maintaining smooth vehicle operations, and closely matching the ground truth metrics of time-to-collision (TTC), headway, and comfort. Notably, the model achieved a significant reduction in fuel consumption, lowering it by 10.42\% compared to actual driving scenarios. These results underscore the capability of RL-based models like EcoFollower to enhance autonomous vehicle algorithms, promoting safer and more energy-efficient driving strategies. △ Less

Submitted 22 July, 2024; originally announced August 2024.

Multi-task SAR Image Processing via GAN-based Unsupervised Manipulation

Authors: Xuran Hu, Mingzhe Zhu, Ziqiang Xu, Zhenpeng Feng, Ljubisa Stankovic

Abstract: Generative Adversarial Networks (GANs) have shown tremendous potential in synthesizing a large number of realistic SAR images by learning patterns in the data distribution. Some GANs can achieve image editing by introducing latent codes, demonstrating significant promise in SAR image processing. Compared to traditional SAR image processing methods, editing based on GAN latent space control is enti… ▽ More Generative Adversarial Networks (GANs) have shown tremendous potential in synthesizing a large number of realistic SAR images by learning patterns in the data distribution. Some GANs can achieve image editing by introducing latent codes, demonstrating significant promise in SAR image processing. Compared to traditional SAR image processing methods, editing based on GAN latent space control is entirely unsupervised, allowing image processing to be conducted without any labeled data. Additionally, the information extracted from the data is more interpretable. This paper proposes a novel SAR image processing framework called GAN-based Unsupervised Editing (GUE), aiming to address the following two issues: (1) disentangling semantic directions in the GAN latent space and finding meaningful directions; (2) establishing a comprehensive SAR image processing framework while achieving multiple image processing functions. In the implementation of GUE, we decompose the entangled semantic directions in the GAN latent space by training a carefully designed network. Moreover, we can accomplish multiple SAR image processing tasks (including despeckling, localization, auxiliary identification, and rotation editing) in a single training process without any form of supervision. Extensive experiments validate the effectiveness of the proposed method. △ Less

Submitted 2 August, 2024; originally announced August 2024.

Comments: 19 pages, 17 figures, 7 tables

Adaptive optical signal-to-noise ratio recovery for long-distance optical fiber transmission

Authors: Mingwen Zhu, Shangsu Ding, Zhixue Li, Song Yu, Jianming Shang, Bin Luo

Abstract: In long-distance fiber optic transmission, the optic fiber link and erbium-doped fiber amplifiers can introduce excessive noise, which reduces the optical signal-to-noise ratio (OSNR). The narrow-band optical filters can be used to eliminate noise and thereby improve OSNR. However, there is a relative frequency drift between the signal and the narrow-band filter, which leads to filtered signal ins… ▽ More In long-distance fiber optic transmission, the optic fiber link and erbium-doped fiber amplifiers can introduce excessive noise, which reduces the optical signal-to-noise ratio (OSNR). The narrow-band optical filters can be used to eliminate noise and thereby improve OSNR. However, there is a relative frequency drift between the signal and the narrow-band filter, which leads to filtered signal instability. This paper proposes an adaptive OSNR recovery scheme based on a Fabry-Perot (F-P) cavity with mode width of 6 MHz. Utilizing the comb filtering of F-P cavity, the noise around the carrier and sidebands of the signal is filtered out simultaneously. To avoid frequency mismatch, we propose a double-servo scheme to suppress relative frequency drift between the signal and the F-P cavity. We constructed a stable radio frequency transfer system based on passive phase compensation and compared our scheme with other OSNR recovery schemes based on optical filters. Compared to the schemes based on dense wavelength division multiplexing (DWDM) and Waveshaper, our scheme demonstrates an improvement in OSNR of carrier by at least 12 dB and sidebands by at least 23.5 dB. The short-term transfer stability (1 s) is improved by one order of magnitude compared to DWDM and half an order of magnitude compared to Waveshper. This scheme can be applied to the recovery of signals with low OSNR in long-distance fiber optic transmission, improving signal quaility and extending the transmission distance limit. △ Less

Submitted 1 August, 2024; originally announced August 2024.

FAST observations of neutral hydrogen in the interacting galaxies NGC 3395/3396

Authors: Nai-Ping Yu, Ming Zhu, Jin-Long Xu, Chuan-Peng Zhang, Hai-Yang Yu, Xiao-Lan Liu, Peng Jiang, Mei Ai

Abstract: We report on high-sensitivity neutral hydrogen observations toward the gas-rich interacting galaxies NGC 3395/3396 with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Compared to previous observations carried out by the Very Large Array (VLA) and the Westerbork Synthesis Radio Telescope (WSRT), a more extended HI envelope around this system has been detected. The total HI gas ma… ▽ More We report on high-sensitivity neutral hydrogen observations toward the gas-rich interacting galaxies NGC 3395/3396 with the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Compared to previous observations carried out by the Very Large Array (VLA) and the Westerbork Synthesis Radio Telescope (WSRT), a more extended HI envelope around this system has been detected. The total HI gas mass of the NGC 3395/3396 system is estimated to be 7.8 x 109 M. This value is 2.7 times more than that reported based on the VLA interferometric maps. Previous observations found a large HI tail extending to the south-west and a minor tail emerging from the north of this peculiar galaxy pair. Based on the high-sensitivity observations of FAST, an extended HI plume to the north-west and a gas plume to the north-east have been detected for the first time. Neutral hydrogen of the two smaller galaxies IC 2604 and IC 2608 on the south of the system have also been detected. We discuss the origins of these extra gas and possible tidal interactions between these galaxies. NGC 3395/3396's most prominent tidal feature, the south-west tail combined with the new detected north-west plume behaves like a large ring. We suggest the ring might be formed by the previous fly-by interaction between NGC 3395 and NGC 3396 which happened 500 Myr ago. Our study shows that high-sensitivity HI observations are important in revealing low column density gas, which is crucial to a deeper understanding of this interacting system. △ Less

Submitted 30 July, 2024; originally announced July 2024.

BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learning

Authors: Baoyuan Wu, Hongrui Chen, Mingda Zhang, Zihao Zhu, Shaokui Wei, Danni Yuan, Mingli Zhu, Ruotong Wang, Li Liu, Chao Shen

Abstract: As an emerging approach to explore the vulnerability of deep neural networks (DNNs), backdoor learning has attracted increasing interest in recent years, and many seminal backdoor attack and defense algorithms are being developed successively or concurrently, in the status of a rapid arms race. However, mainly due to the diverse settings, and the difficulties of implementation and reproducibility… ▽ More As an emerging approach to explore the vulnerability of deep neural networks (DNNs), backdoor learning has attracted increasing interest in recent years, and many seminal backdoor attack and defense algorithms are being developed successively or concurrently, in the status of a rapid arms race. However, mainly due to the diverse settings, and the difficulties of implementation and reproducibility of existing works, there is a lack of a unified and standardized benchmark of backdoor learning, causing unfair comparisons or unreliable conclusions (e.g., misleading, biased or even false conclusions). Consequently, it is difficult to evaluate the current progress and design the future development roadmap of this literature. To alleviate this dilemma, we build a comprehensive benchmark of backdoor learning called BackdoorBench. Our benchmark makes three valuable contributions to the research community. 1) We provide an integrated implementation of state-of-the-art (SOTA) backdoor learning algorithms (currently including 20 attack and 32 defense algorithms), based on an extensible modular-based codebase. 2) We conduct comprehensive evaluations with 5 poisoning ratios, based on 4 models and 4 datasets, leading to 11,492 pairs of attack-against-defense evaluations in total. 3) Based on above evaluations, we present abundant analysis from 10 perspectives via 18 useful analysis tools, and provide several inspiring insights about backdoor learning. We hope that our efforts could build a solid foundation of backdoor learning to facilitate researchers to investigate existing algorithms, develop more innovative algorithms, and explore the intrinsic mechanism of backdoor learning. Finally, we have created a user-friendly website at http://backdoorbench.com, which collects all important information of BackdoorBench, including codebase, docs, leaderboard, and model Zoo. △ Less

Submitted 29 July, 2024; originally announced July 2024.

Comments: Substantial extensions based on our previous conference version "Backdoorbench: A comprehensive benchmark of backdoor learning" published at NeurIPS D&B Track 2022. 20 backdoor attack algorithms, 32 backdoor defense algorithms, 11000+ pairs of attack-against-defense evaluations, 10 analyses, 18 analysis tools

Is the Digital Forensics and Incident Response Pipeline Ready for Text-Based Threats in LLM Era?

Authors: Avanti Bhandarkar, Ronald Wilson, Anushka Swarup, Mengdi Zhu, Damon Woodard

Abstract: In the era of generative AI, the widespread adoption of Neural Text Generators (NTGs) presents new cybersecurity challenges, particularly within the realms of Digital Forensics and Incident Response (DFIR). These challenges primarily involve the detection and attribution of sources behind advanced attacks like spearphishing and disinformation campaigns. As NTGs evolve, the task of distinguishing b… ▽ More In the era of generative AI, the widespread adoption of Neural Text Generators (NTGs) presents new cybersecurity challenges, particularly within the realms of Digital Forensics and Incident Response (DFIR). These challenges primarily involve the detection and attribution of sources behind advanced attacks like spearphishing and disinformation campaigns. As NTGs evolve, the task of distinguishing between human and NTG-authored texts becomes critically complex. This paper rigorously evaluates the DFIR pipeline tailored for text-based security systems, specifically focusing on the challenges of detecting and attributing authorship of NTG-authored texts. By introducing a novel human-NTG co-authorship text attack, termed CS-ACT, our study uncovers significant vulnerabilities in traditional DFIR methodologies, highlighting discrepancies between ideal scenarios and real-world conditions. Utilizing 14 diverse datasets and 43 unique NTGs, up to the latest GPT-4, our research identifies substantial vulnerabilities in the forensic profiling phase, particularly in attributing authorship to NTGs. Our comprehensive evaluation points to factors such as model sophistication and the lack of distinctive style within NTGs as significant contributors for these vulnerabilities. Our findings underscore the necessity for more sophisticated and adaptable strategies, such as incorporating adversarial learning, stylizing NTGs, and implementing hierarchical attribution through the mapping of NTG lineages to enhance source attribution. This sets the stage for future research and the development of more resilient text-based security systems. △ Less

Submitted 25 July, 2024; originally announced July 2024.

Comments: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

SE3ET: SE(3)-Equivariant Transformer for Low-Overlap Point Cloud Registration

Authors: Chien Erh Lin, Minghan Zhu, Maani Ghaffari

Abstract: Partial point cloud registration is a challenging problem in robotics, especially when the robot undergoes a large transformation, causing a significant initial pose error and a low overlap between measurements. This work proposes exploiting equivariant learning from 3D point clouds to improve registration robustness. We propose SE3ET, an SE(3)-equivariant registration framework that employs equiv… ▽ More Partial point cloud registration is a challenging problem in robotics, especially when the robot undergoes a large transformation, causing a significant initial pose error and a low overlap between measurements. This work proposes exploiting equivariant learning from 3D point clouds to improve registration robustness. We propose SE3ET, an SE(3)-equivariant registration framework that employs equivariant point convolution and equivariant transformer designs to learn expressive and robust geometric features. We tested the proposed registration method on indoor and outdoor benchmarks where the point clouds are under arbitrary transformations and low overlapping ratios. We also provide generalization tests and run-time performance. △ Less

Submitted 23 July, 2024; originally announced July 2024.

FASHI: A blind survey of 21cm HI absorption galaxies with FAST

Authors: Chuan-Peng Zhang, Ming Zhu, Peng Jiang, Cheng Cheng, Jin-Long Xu, Nai-Ping Yu, Xiao-Lan Liu, Bo Zhang

Abstract: The FAST All Sky HI survey (FASHI) is broader in frequency band and deeper in detection sensitivity than most of previous HI surveys. FASHI is designed to cover the entire sky observable by the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Based on the FASHI data, we perform a blind survey of 21cm HI absorption galaxies at redshift $z<0.09$ over an area of about 10000 square degree… ▽ More The FAST All Sky HI survey (FASHI) is broader in frequency band and deeper in detection sensitivity than most of previous HI surveys. FASHI is designed to cover the entire sky observable by the Five-hundred-meter Aperture Spherical radio Telescope (FAST). Based on the FASHI data, we perform a blind survey of 21cm HI absorption galaxies at redshift $z<0.09$ over an area of about 10000 square degrees. We detected 51 HI absorbers, including 21 previously known and 30 new ones, with 8 sources having no optical spectroscopic redshift. The probability of occurrence for HI absorbers in all HI galaxies is 1/1078. The radio flux densities of the FASHI absorbers are mainly concentrated in the range of $S_{\rm 1.4GHz}=10\sim100$ mJy, but even as low as $2.6\pm0.4$mJy. The number of redshifted absorbers is slightly higher than the number of blueshifted absorbers. Such results would provide some important clues for future flux-selected HI absorber surveys. Therefore, FAST has significantly improved the capabilities and performance for HI absorption observations and provided a true blind survey of 21cm HI absorption galaxies for such studies. △ Less

Submitted 22 July, 2024; originally announced July 2024.

Comments: 32 pages, 7 figures, 2 tables, submitted to ApJS in 05/24/2024, Comments are welcome

A Novel Hybrid Digital and Analog Laser Synchronization System

Authors: Mingwen Zhu, Shangsu Ding, Tianwei Jiang, Jianming Shang, Song Yu, Bin Luo

Abstract: Laser synchronization is a technique that locks the wavelength of a free-running laser to that of the reference laser, thereby enabling synchronous changes in the wavelengths of the two lasers. This technique is of crucial importance in both scientific and industrial applications. Conventional synchronization systems, whether digital or analog, have intrinsic limitations in terms of accuracy or ba… ▽ More Laser synchronization is a technique that locks the wavelength of a free-running laser to that of the reference laser, thereby enabling synchronous changes in the wavelengths of the two lasers. This technique is of crucial importance in both scientific and industrial applications. Conventional synchronization systems, whether digital or analog, have intrinsic limitations in terms of accuracy or bandwidth. The hybrid "digital + analog" system can address this shortcoming. However, all above systems face the challenge of achieving an both high locking accuracy and low structural complexity simultaneously. This paper presents a hybrid "digital + analog" laser synchronization system with low-complexity and high-performance. In the digital part, we proposed a electric intensity locking method based on a band-pass filter, which realizes the fluctuation of frequency offset between a single frequency laser (SFL) and a mode-locked laser (MLL) less than 350 kHz in 24 hours. Following the incorporation of the analog control component, frequency fluctuation is less than 2.5 Hz in 24 hours. By synchronizing two SFLs to a repetition-frequency locked MLL, we achieve indirect synchronization between SFLs with a frequency offset of 10.6 GHz and fluctuation less than 5 Hz in 24 hours, demonstrating robust long- and short-term stability. Since the MLL is employed as a reference, the system can be utilized for cross-band indirect synchronization of multiple lasers. Based on the synchronization system, we propose a photonic-assisted microwave frequency identification scheme, which has detection error of less than 0.6 MHz. The high performance of the synchronization system enables the proposed frequency identification scheme to achieve high measurement accuracy and a theoretically large frequency range. △ Less

Submitted 21 July, 2024; originally announced July 2024.

Comments: 7 pages,9 figures

The FAST HI 21-cm absorption blind survey. II -- statistic exploration for associated and intervening systems

Authors: Wenkai Hu, Yougang Wang, Yichao Li, Ue-Li Pen, Jie Wang, Yingjie Jing, Ming Zhu, Xin Zhang, Wenxiu Yang, Yidong Xu, Xu Chen, Jingze Chen, Zheng Zheng, Di Li, Xuelei Chen

Abstract: We present an extragalactic HI 21-cm absorption lines catalog from a blind search at z $\leq$ 0.35, using drift-scan data collected in 1616.9 hours by the ongoing Commensal Radio Astronomy FasT Survey (CRAFTS) and FAST All Sky HI Survey (FASHI), which spans a sky area of 7456.8 deg$^{2}$ and covers 84,533 radio sources with a flux density greater than 12 mJy. 14 previously identified HI absorbers… ▽ More We present an extragalactic HI 21-cm absorption lines catalog from a blind search at z $\leq$ 0.35, using drift-scan data collected in 1616.9 hours by the ongoing Commensal Radio Astronomy FasT Survey (CRAFTS) and FAST All Sky HI Survey (FASHI), which spans a sky area of 7456.8 deg$^{2}$ and covers 84,533 radio sources with a flux density greater than 12 mJy. 14 previously identified HI absorbers and 20 newly discovered HI absorbers were detected, comprising 14 associated systems, 11 intervening systems, and 9 systems with undetermined classifications. We fit HI profiles with multi-component Gaussian functions and calculate the redshift, width, flux density, optical depth, and HI column densities for each source. Through spectral stacking, the mean peak optical path, mean velocity-integrated optical path $\langle τ\rangle$, mean FWHM and mean HI column density $\langle$ N$_{HI}\rangle$ are measured to be 0.46 and 0.34; 25.85 km/s and 4.62 km/s; 39.80 km/s and 8.95 km/s; 0.470 and 0.085 T$_{s} \times$ 10$^{20}$cm$^{-2}$K$^{-1}$, for the associated and intervening samples, respectively. Statistical analysis also reveals that associated systems tend to be hosted by red (g$-$r$>$0.7) galaxies at lower redshifts, whereas galaxies hosting intervening HI absorption are typically found at higher redshifts and are of a bluer (g$-$r$\leq$0.7) type. Additionally, it has been demonstrated that associated HI 21-cm absorptions connected to compact radio sources display higher N$_{HI}$ values compared to those linked with extended radio sources. △ Less

Submitted 19 July, 2024; originally announced July 2024.

Comments: 28 pages, 39 figures, 5 tables

Continual Learning for Adaptable Car-Following in Dynamic Traffic Environments

Authors: Xianda Chen, PakHin Tiu, Xu Han, Junjie Chen, Yuanfei Wu, Xinhu Zheng, Meixin Zhu

Abstract: The continual evolution of autonomous driving technology requires car-following models that can adapt to diverse and dynamic traffic environments. Traditional learning-based models often suffer from performance degradation when encountering unseen traffic patterns due to a lack of continual learning capabilities. This paper proposes a novel car-following model based on continual learning that addr… ▽ More The continual evolution of autonomous driving technology requires car-following models that can adapt to diverse and dynamic traffic environments. Traditional learning-based models often suffer from performance degradation when encountering unseen traffic patterns due to a lack of continual learning capabilities. This paper proposes a novel car-following model based on continual learning that addresses this limitation. Our framework incorporates Elastic Weight Consolidation (EWC) and Memory Aware Synapses (MAS) techniques to mitigate catastrophic forgetting and enable the model to learn incrementally from new traffic data streams. We evaluate the performance of the proposed model on the Waymo and Lyft datasets which encompass various traffic scenarios. The results demonstrate that the continual learning techniques significantly outperform the baseline model, achieving 0\% collision rates across all traffic conditions. This research contributes to the advancement of autonomous driving technology by fostering the development of more robust and adaptable car-following models. △ Less

Submitted 17 July, 2024; originally announced July 2024.

Causal Inference with Complex Treatments: A Survey

Authors: Yingrong Wang, Haoxuan Li, Minqin Zhu, Anpeng Wu, Ruoxuan Xiong, Fei Wu, Kun Kuang

Abstract: Causal inference plays an important role in explanatory analysis and decision making across various fields like statistics, marketing, health care, and education. Its main task is to estimate treatment effects and make intervention policies. Traditionally, most of the previous works typically focus on the binary treatment setting that there is only one treatment for a unit to adopt or not. However… ▽ More Causal inference plays an important role in explanatory analysis and decision making across various fields like statistics, marketing, health care, and education. Its main task is to estimate treatment effects and make intervention policies. Traditionally, most of the previous works typically focus on the binary treatment setting that there is only one treatment for a unit to adopt or not. However, in practice, the treatment can be much more complex, encompassing multi-valued, continuous, or bundle options. In this paper, we refer to these as complex treatments and systematically and comprehensively review the causal inference methods for addressing them. First, we formally revisit the problem definition, the basic assumptions, and their possible variations under specific conditions. Second, we sequentially review the related methods for multi-valued, continuous, and bundled treatment settings. In each situation, we tentatively divide the methods into two categories: those conforming to the unconfoundedness assumption and those violating it. Subsequently, we discuss the available datasets and open-source codes. Finally, we provide a brief summary of these works and suggest potential directions for future research. △ Less

Submitted 19 July, 2024; originally announced July 2024.

LogoSticker: Inserting Logos into Diffusion Models for Customized Generation

Authors: Mingkang Zhu, Xi Chen, Zhongdao Wang, Hengshuang Zhao, Jiaya Jia

Abstract: Recent advances in text-to-image model customization have underscored the importance of integrating new concepts with a few examples. Yet, these progresses are largely confined to widely recognized subjects, which can be learned with relative ease through models' adequate shared prior knowledge. In contrast, logos, characterized by unique patterns and textual elements, are hard to establish shared… ▽ More Recent advances in text-to-image model customization have underscored the importance of integrating new concepts with a few examples. Yet, these progresses are largely confined to widely recognized subjects, which can be learned with relative ease through models' adequate shared prior knowledge. In contrast, logos, characterized by unique patterns and textual elements, are hard to establish shared knowledge within diffusion models, thus presenting a unique challenge. To bridge this gap, we introduce the task of logo insertion. Our goal is to insert logo identities into diffusion models and enable their seamless synthesis in varied contexts. We present a novel two-phase pipeline LogoSticker to tackle this task. First, we propose the actor-critic relation pre-training algorithm, which addresses the nontrivial gaps in models' understanding of the potential spatial positioning of logos and interactions with other objects. Second, we propose a decoupled identity learning algorithm, which enables precise localization and identity extraction of logos. LogoSticker can generate logos accurately and harmoniously in diverse contexts. We comprehensively validate the effectiveness of LogoSticker over customization methods and large models such as DALLE~3. \href{https://mingkangz.github.io/logosticker}{Project page}. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Comments: ECCV2024

SciCode: A Research Coding Benchmark Curated by Scientists

Authors: Minyang Tian, Luyu Gao, Shizhuo Dylan Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, Shengyan Liu, Di Luo, Yutao Ma, Hao Tong, Kha Trinh, Chenyu Tian, Zihan Wang, Bohao Wu, Yanyu Xiong, Shengzhu Yin, Minhui Zhu, Kilian Lieret, Yanxin Lu, Genglin Liu, Yufeng Du , et al. (5 additional authors not shown)

Abstract: Since language models (LMs) now outperform average humans on many challenging tasks, it has become increasingly difficult to develop challenging, high-quality, and realistic evaluations. We address this issue by examining LMs' capabilities to generate code for solving real scientific research problems. Incorporating input from scientists and AI researchers in 16 diverse natural science sub-fields,… ▽ More Since language models (LMs) now outperform average humans on many challenging tasks, it has become increasingly difficult to develop challenging, high-quality, and realistic evaluations. We address this issue by examining LMs' capabilities to generate code for solving real scientific research problems. Incorporating input from scientists and AI researchers in 16 diverse natural science sub-fields, including mathematics, physics, chemistry, biology, and materials science, we created a scientist-curated coding benchmark, SciCode. The problems in SciCode naturally factorize into multiple subproblems, each involving knowledge recall, reasoning, and code synthesis. In total, SciCode contains 338 subproblems decomposed from 80 challenging main problems. It offers optional descriptions specifying useful scientific background information and scientist-annotated gold-standard solutions and test cases for evaluation. Claude3.5-Sonnet, the best-performing model among those tested, can solve only 4.6% of the problems in the most realistic setting. We believe that SciCode demonstrates both contemporary LMs' progress towards becoming helpful scientific assistants and sheds light on the development and evaluation of scientific AI in the future. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Comments: 25 pages, 9 figures, 7 tables

Application of Prompt Learning Models in Identifying the Collaborative Problem Solving Skills in an Online Task

Authors: Mengxiao Zhu, Xin Wang, Xiantao Wang, Zihang Chen, Wei Huang

Abstract: Collaborative problem solving (CPS) competence is considered one of the essential 21st-century skills. To facilitate the assessment and learning of CPS competence, researchers have proposed a series of frameworks to conceptualize CPS and explored ways to make sense of the complex processes involved in collaborative problem solving. However, encoding explicit behaviors into subskills within the fra… ▽ More Collaborative problem solving (CPS) competence is considered one of the essential 21st-century skills. To facilitate the assessment and learning of CPS competence, researchers have proposed a series of frameworks to conceptualize CPS and explored ways to make sense of the complex processes involved in collaborative problem solving. However, encoding explicit behaviors into subskills within the frameworks of CPS skills is still a challenging task. Traditional studies have relied on manual coding to decipher behavioral data for CPS, but such coding methods can be very time-consuming and cannot support real-time analyses. Scholars have begun to explore approaches for constructing automatic coding models. Nevertheless, the existing models built using machine learning or deep learning techniques depend on a large amount of training data and have relatively low accuracy. To address these problems, this paper proposes a prompt-based learning pre-trained model. The model can achieve high performance even with limited training data. In this study, three experiments were conducted, and the results showed that our model not only produced the highest accuracy, macro F1 score, and kappa values on large training sets, but also performed the best on small training sets of the CPS behavioral data. The application of the proposed prompt-based learning pre-trained model contributes to the CPS skills coding task and can also be used for other CSCW coding tasks to replace manual coding. △ Less

Submitted 17 July, 2024; originally announced July 2024.

Min-max theory and existence of H-spheres with arbitrary codimensions

Authors: Rui Gao, Miaomiao Zhu

Abstract: We demonstrate the existence of branched immersed 2-spheres with prescribed mean curvature, with controlled Morse index and with arbitrary codimensions in closed Riemannian manifold $N$ admitting finite fundamental group, where $π_k(N) \neq 0$ and $k \geq 2$, for certain generic choice of prescribed mean curvature vector. Moreover, we enhance this existence result to encompass all possible choices… ▽ More We demonstrate the existence of branched immersed 2-spheres with prescribed mean curvature, with controlled Morse index and with arbitrary codimensions in closed Riemannian manifold $N$ admitting finite fundamental group, where $π_k(N) \neq 0$ and $k \geq 2$, for certain generic choice of prescribed mean curvature vector. Moreover, we enhance this existence result to encompass all possible choices of prescribed mean curvatures under certain Ricci curvature condition on $N$ when $\dim{N} = 3$. When $\dim{N} \geq 4$, we establish a Morse index lower bound while $N$ satisfies some isotropic curvature condition. As a consequence, we can leverage latter strengthened result to construct 2-spheres with parallel mean curvature when $N$ has positive isotropic curvature and $\dim{N} \geq 4$. At last, we partially resolve the homotopy problem concerning the existence of a representative surface with prescribed mean curvature type vector field in some given homotopy classes. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Comments: 137 pages. Comments Welcome!

MSC Class: 49J35; 53A10; 53C42; 58E20

Bridging Data Gaps in Healthcare: A Scoping Review of Transfer Learning in Biomedical Data Analysis

Authors: Siqi Li, Xin Li, Kunyu Yu, Di Miao, Mingcheng Zhu, Mengying Yan, Yuhe Ke, Danny D'Agostino, Yilin Ning, Qiming Wu, Ziwen Wang, Yuqing Shang, Molei Liu, Chuan Hong, Nan Liu

Abstract: Clinical and biomedical research in low-resource settings often faces significant challenges due to the need for high-quality data with sufficient sample sizes to construct effective models. These constraints hinder robust model training and prompt researchers to seek methods for leveraging existing knowledge from related studies to support new research efforts. Transfer learning (TL), a machine l… ▽ More Clinical and biomedical research in low-resource settings often faces significant challenges due to the need for high-quality data with sufficient sample sizes to construct effective models. These constraints hinder robust model training and prompt researchers to seek methods for leveraging existing knowledge from related studies to support new research efforts. Transfer learning (TL), a machine learning technique, emerges as a powerful solution by utilizing knowledge from pre-trained models to enhance the performance of new models, offering promise across various healthcare domains. Despite its conceptual origins in the 1990s, the application of TL in medical research has remained limited, especially beyond image analysis. In our review of TL applications in structured clinical and biomedical data, we screened 3,515 papers, with 55 meeting the inclusion criteria. Among these, only 2% (one out of 55) utilized external studies, and 7% (four out of 55) addressed scenarios involving multi-site collaborations with privacy constraints. To achieve actionable TL with structured medical data while addressing regional disparities, inequality, and privacy constraints in healthcare research, we advocate for the careful identification of appropriate source data and models, the selection of suitable TL frameworks, and the validation of TL models with proper baselines. △ Less

Submitted 4 July, 2024; originally announced July 2024.

Incremental Multiview Point Cloud Registration with Two-stage Candidate Retrieval

Authors: Shiqi Li, Jihua Zhu, Yifan Xie, Mingchen Zhu

Abstract: Multiview point cloud registration serves as a cornerstone of various computer vision tasks. Previous approaches typically adhere to a global paradigm, where a pose graph is initially constructed followed by motion synchronization to determine the absolute pose. However, this separated approach may not fully leverage the characteristics of multiview registration and might struggle with low-overlap… ▽ More Multiview point cloud registration serves as a cornerstone of various computer vision tasks. Previous approaches typically adhere to a global paradigm, where a pose graph is initially constructed followed by motion synchronization to determine the absolute pose. However, this separated approach may not fully leverage the characteristics of multiview registration and might struggle with low-overlap scenarios. In this paper, we propose an incremental multiview point cloud registration method that progressively registers all scans to a growing meta-shape. To determine the incremental ordering, we employ a two-stage coarse-to-fine strategy for point cloud candidate retrieval. The first stage involves the coarse selection of scans based on neighbor fusion-enhanced global aggregation features, while the second stage further reranks candidates through geometric-based matching. Additionally, we apply a transformation averaging technique to mitigate accumulated errors during the registration process. Finally, we utilize a Reservoir sampling-based technique to address density variance issues while reducing computational load. Comprehensive experimental results across various benchmarks validate the effectiveness and generalization of our approach. △ Less

Submitted 10 July, 2024; originally announced July 2024.

GenFollower: Enhancing Car-Following Prediction with Large Language Models

Authors: Xianda Chen, Mingxing Peng, PakHin Tiu, Yuanfei Wu, Junjie Chen, Meixin Zhu, Xinhu Zheng

Abstract: Accurate modeling of car-following behaviors is essential for various applications in traffic management and autonomous driving systems. However, current approaches often suffer from limitations like high sensitivity to data quality and lack of interpretability. In this study, we propose GenFollower, a novel zero-shot prompting approach that leverages large language models (LLMs) to address these… ▽ More Accurate modeling of car-following behaviors is essential for various applications in traffic management and autonomous driving systems. However, current approaches often suffer from limitations like high sensitivity to data quality and lack of interpretability. In this study, we propose GenFollower, a novel zero-shot prompting approach that leverages large language models (LLMs) to address these challenges. We reframe car-following behavior as a language modeling problem and integrate heterogeneous inputs into structured prompts for LLMs. This approach achieves improved prediction performance and interpretability compared to traditional baseline models. Experiments on the Waymo Open datasets demonstrate GenFollower's superior performance and ability to provide interpretable insights into factors influencing car-following behavior. This work contributes to advancing the understanding and prediction of car-following behaviors, paving the way for enhanced traffic management and autonomous driving systems. △ Less

Submitted 8 July, 2024; originally announced July 2024.

A Survey of Models for Cognitive Diagnosis: New Developments and Future Directions

Authors: Fei Wang, Weibo Gao, Qi Liu, Jiatong Li, Guanhao Zhao, Zheng Zhang, Zhenya Huang, Mengxiao Zhu, Shijin Wang, Wei Tong, Enhong Chen

Abstract: Cognitive diagnosis has been developed for decades as an effective measurement tool to evaluate human cognitive status such as ability level and knowledge mastery. It has been applied to a wide range of fields including education, sport, psychological diagnosis, etc. By providing better awareness of cognitive status, it can serve as the basis for personalized services such as well-designed medical… ▽ More Cognitive diagnosis has been developed for decades as an effective measurement tool to evaluate human cognitive status such as ability level and knowledge mastery. It has been applied to a wide range of fields including education, sport, psychological diagnosis, etc. By providing better awareness of cognitive status, it can serve as the basis for personalized services such as well-designed medical treatment, teaching strategy and vocational training. This paper aims to provide a survey of current models for cognitive diagnosis, with more attention on new developments using machine learning-based methods. By comparing the model structures, parameter estimation algorithms, model evaluation methods and applications, we provide a relatively comprehensive review of the recent trends in cognitive diagnosis models. Further, we discuss future directions that are worthy of exploration. In addition, we release two Python libraries: EduData for easy access to some relevant public datasets we have collected, and EduCDM that implements popular CDMs to facilitate both applications and research purposes. △ Less

Submitted 7 July, 2024; originally announced July 2024.

Unsupervised 4D Cardiac Motion Tracking with Spatiotemporal Optical Flow Networks

Authors: Long Teng, Wei Feng, Menglong Zhu, Xinchao Li

Abstract: Cardiac motion tracking from echocardiography can be used to estimate and quantify myocardial motion within a cardiac cycle. It is a cost-efficient and effective approach for assessing myocardial function. However, ultrasound imaging has the inherent characteristics of spatially low resolution and temporally random noise, which leads to difficulties in obtaining reliable annotation. Thus it is dif… ▽ More Cardiac motion tracking from echocardiography can be used to estimate and quantify myocardial motion within a cardiac cycle. It is a cost-efficient and effective approach for assessing myocardial function. However, ultrasound imaging has the inherent characteristics of spatially low resolution and temporally random noise, which leads to difficulties in obtaining reliable annotation. Thus it is difficult to perform supervised learning for motion tracking. In addition, there is no end-to-end unsupervised method currently in the literature. This paper presents a motion tracking method where unsupervised optical flow networks are designed with spatial reconstruction loss and temporal-consistency loss. Our proposed loss functions make use of the pair-wise and temporal correlation to estimate cardiac motion from noisy background. Experiments using a synthetic 4D echocardiography dataset has shown the effectiveness of our approach, and its superiority over existing methods on both accuracy and running speed. To the best of our knowledge, this is the first work performed that uses unsupervised end-to-end deep learning optical flow network for 4D cardiac motion tracking. △ Less

Submitted 5 July, 2024; originally announced July 2024.

CAV-AHDV-CAV: Mitigating Traffic Oscillations for CAVs through a Novel Car-Following Structure and Reinforcement Learning

Authors: Xianda Chen, PakHin Tiu, Yihuai Zhang, Xinhu Zheng, Meixin Zhu

Abstract: Connected and Automated Vehicles (CAVs) offer a promising solution to the challenges of mixed traffic with both CAVs and Human-Driven Vehicles (HDVs). A significant hurdle in such scenarios is traffic oscillation, or the "stop-and-go" pattern, during car-following situations. While HDVs rely on limited information, CAVs can leverage data from other CAVs for better decision-making. This allows CAVs… ▽ More Connected and Automated Vehicles (CAVs) offer a promising solution to the challenges of mixed traffic with both CAVs and Human-Driven Vehicles (HDVs). A significant hurdle in such scenarios is traffic oscillation, or the "stop-and-go" pattern, during car-following situations. While HDVs rely on limited information, CAVs can leverage data from other CAVs for better decision-making. This allows CAVs to anticipate and mitigate the spread of deceleration waves that worsen traffic flow. We propose a novel "CAV-AHDV-CAV" car-following framework that treats the sequence of HDVs between two CAVs as a single entity, eliminating noise from individual driver behaviors. This deep reinforcement learning approach analyzes vehicle equilibrium states and employs a state fusion strategy. Trained and tested on diverse datasets (HighD, NGSIM, SPMD, Waymo, Lyft) encompassing over 70,000 car-following instances, our model outperforms baselines in collision avoidance, maintaining equilibrium with both preceding and leading vehicles and achieving the lowest standard deviation of time headway. These results demonstrate the effectiveness of our approach in developing robust CAV control strategies for mixed traffic. Our model has the potential to mitigate traffic oscillation, improve traffic flow efficiency, and enhance overall safety. △ Less

Submitted 10 July, 2024; v1 submitted 23 June, 2024; originally announced July 2024.

Comments: Due to errors identified in the experimental methodology and results sections, we request to withdraw this paper

EditFollower: Tunable Car Following Models for Customizable Adaptive Cruise Control Systems

Authors: Xianda Chen, Xu Han, Meixin Zhu, Xiaowen Chu, PakHin Tiu, Xinhu Zheng, Yinhai Wang

Abstract: In the realm of driving technologies, fully autonomous vehicles have not been widely adopted yet, making advanced driver assistance systems (ADAS) crucial for enhancing driving experiences. Adaptive Cruise Control (ACC) emerges as a pivotal component of ADAS. However, current ACC systems often employ fixed settings, failing to intuitively capture drivers' social preferences and leading to potentia… ▽ More In the realm of driving technologies, fully autonomous vehicles have not been widely adopted yet, making advanced driver assistance systems (ADAS) crucial for enhancing driving experiences. Adaptive Cruise Control (ACC) emerges as a pivotal component of ADAS. However, current ACC systems often employ fixed settings, failing to intuitively capture drivers' social preferences and leading to potential function disengagement. To overcome these limitations, we propose the Editable Behavior Generation (EBG) model, a data-driven car-following model that allows for adjusting driving discourtesy levels. The framework integrates diverse courtesy calculation methods into long short-term memory (LSTM) and Transformer architectures, offering a comprehensive approach to capture nuanced driving dynamics. By integrating various discourtesy values during the training process, our model generates realistic agent trajectories with different levels of courtesy in car-following behavior. Experimental results on the HighD and Waymo datasets showcase a reduction in Mean Squared Error (MSE) of spacing and MSE of speed compared to baselines, establishing style controllability. To the best of our knowledge, this work represents the first data-driven car-following model capable of dynamically adjusting discourtesy levels. Our model provides valuable insights for the development of ACC systems that take into account drivers' social preferences. △ Less

Submitted 23 June, 2024; originally announced July 2024.

CausalPrism: A Visual Analytics Approach for Subgroup-based Causal Heterogeneity Exploration

Authors: Jiehui Zhou, Xumeng Wang, Kam-Kwai Wong, Wei Zhang, Xingyu Liu, Juntian Zhang, Minfeng Zhu, Wei Chen

Abstract: In causal inference, estimating Heterogeneous Treatment Effects (HTEs) from observational data is critical for understanding how different subgroups respond to treatments, with broad applications such as precision medicine and targeted advertising. However, existing work on HTE, subgroup discovery, and causal visualization is insufficient to address two challenges: first, the sheer number of poten… ▽ More In causal inference, estimating Heterogeneous Treatment Effects (HTEs) from observational data is critical for understanding how different subgroups respond to treatments, with broad applications such as precision medicine and targeted advertising. However, existing work on HTE, subgroup discovery, and causal visualization is insufficient to address two challenges: first, the sheer number of potential subgroups and the necessity to balance multiple objectives (e.g., high effects and low variances) pose a considerable analytical challenge. Second, effective subgroup analysis has to follow the analysis goal specified by users and provide causal results with verification. To this end, we propose a visual analytics approach for subgroup-based causal heterogeneity exploration. Specifically, we first formulate causal subgroup discovery as a constrained multi-objective optimization problem and adopt a heuristic genetic algorithm to learn the Pareto front of optimal subgroups described by interpretable rules. Combining with this model, we develop a prototype system, CausalPrism, that incorporates tabular visualization, multi-attribute rankings, and uncertainty plots to support users in interactively exploring and sorting subgroups and explaining treatment effects. Quantitative experiments validate that the proposed model can efficiently mine causal subgroups that outperform state-of-the-art HTE and subgroup discovery methods, and case studies and expert interviews demonstrate the effectiveness and usability of the system. Code is available at https://osf.io/jaqmf/?view_only=ac9575209945476b955bf829c85196e9. △ Less

Submitted 11 August, 2024; v1 submitted 1 July, 2024; originally announced July 2024.

Comments: 12 pages, 7 figures

Active-RIS-Aided Covert Communications in NOMA-Inspired ISAC Wireless Systems

Authors: Miaomiao Zhu, Pengxu Chen, Liang Yang, Alexandros-Apostolos A. Boulogeorgos, Theodoros A. Tsiftsis, Hongwu Liu

Abstract: Non-orthogonal multiple access (NOMA)-inspired integrated sensing and communication (ISAC) facilitates spectrum sharing for radar sensing and NOMA communications, whereas facing privacy and security challenges due to open wireless propagation. In this paper, active reconfigurable intelligent surface (RIS) is employed to aid covert communications in NOMA-inspired ISAC wireless system with the aim o… ▽ More Non-orthogonal multiple access (NOMA)-inspired integrated sensing and communication (ISAC) facilitates spectrum sharing for radar sensing and NOMA communications, whereas facing privacy and security challenges due to open wireless propagation. In this paper, active reconfigurable intelligent surface (RIS) is employed to aid covert communications in NOMA-inspired ISAC wireless system with the aim of maximizing the covert rate. Specifically, a dual-function base-station (BS) transmits the superposition signal to sense multiple targets, while achieving covert and reliable communications for a pair of NOMA covert and public users, respectively, in the presence of a warden. Two superposition transmission schemes, namely, the transmissions with dedicated sensing signal (w-DSS) and without dedicated sensing signal (w/o-DSS), are respectively considered in the formulations of the joint transmission and reflection beamforming optimization problems. Numerical results demonstrate that active-RIS-aided NOMA-ISAC system outperforms the passive-RIS-aided and without-RIS counterparts in terms of covert rate and trade-off between covert communication and sensing performance metrics. Finally, the w/o-DSS scheme, which omits the dedicated sensing signal, achieves a higher covert rate than the w-DSS scheme by allocating more transmit power for the covert transmissions, while preserving a comparable multi-target sensing performance. △ Less

Submitted 29 June, 2024; originally announced July 2024.

Simulating Financial Market via Large Language Model based Agents

Authors: Shen Gao, Yuntao Wen, Minghang Zhu, Jianing Wei, Yuhan Cheng, Qunzi Zhang, Shuo Shang

Abstract: Most economic theories typically assume that financial market participants are fully rational individuals and use mathematical models to simulate human behavior in financial markets. However, human behavior is often not entirely rational and is challenging to predict accurately with mathematical models. In this paper, we propose \textbf{A}gent-based \textbf{S}imulated \textbf{F}inancial \textbf{M}… ▽ More Most economic theories typically assume that financial market participants are fully rational individuals and use mathematical models to simulate human behavior in financial markets. However, human behavior is often not entirely rational and is challenging to predict accurately with mathematical models. In this paper, we propose \textbf{A}gent-based \textbf{S}imulated \textbf{F}inancial \textbf{M}arket (ASFM), which first constructs a simulated stock market with a real order matching system. Then, we propose a large language model based agent as the stock trader, which contains the profile, observation, and tool-learning based action module. The trading agent can comprehensively understand current market dynamics and financial policy information, and make decisions that align with their trading strategy. In the experiments, we first verify that the reactions of our ASFM are consistent with the real stock market in two controllable scenarios. In addition, we also conduct experiments in two popular economics research directions, and we find that conclusions drawn in our \model align with the preliminary findings in economics research. Based on these observations, we believe our proposed ASFM provides a new paradigm for economic research. △ Less

Submitted 28 June, 2024; originally announced June 2024.

MMRo: Are Multimodal LLMs Eligible as the Brain for In-Home Robotics?

Authors: Jinming Li, Yichen Zhu, Zhiyuan Xu, Jindong Gu, Minjie Zhu, Xin Liu, Ning Liu, Yaxin Peng, Feifei Feng, Jian Tang

Abstract: It is fundamentally challenging for robots to serve as useful assistants in human environments because this requires addressing a spectrum of sub-problems across robotics, including perception, language understanding, reasoning, and planning. The recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated their exceptional abilities in solving complex mathematical problems, m… ▽ More It is fundamentally challenging for robots to serve as useful assistants in human environments because this requires addressing a spectrum of sub-problems across robotics, including perception, language understanding, reasoning, and planning. The recent advancements in Multimodal Large Language Models (MLLMs) have demonstrated their exceptional abilities in solving complex mathematical problems, mastering commonsense and abstract reasoning. This has led to the recent utilization of MLLMs as the brain in robotic systems, enabling these models to conduct high-level planning prior to triggering low-level control actions for task execution. However, it remains uncertain whether existing MLLMs are reliable in serving the brain role of robots. In this study, we introduce the first benchmark for evaluating Multimodal LLM for Robotic (MMRo) benchmark, which tests the capability of MLLMs for robot applications. Specifically, we identify four essential capabilities perception, task planning, visual reasoning, and safety measurement that MLLMs must possess to qualify as the robot's central processing unit. We have developed several scenarios for each capability, resulting in a total of 14 metrics for evaluation. We present experimental results for various MLLMs, including both commercial and open-source models, to assess the performance of existing systems. Our findings indicate that no single model excels in all areas, suggesting that current MLLMs are not yet trustworthy enough to serve as the cognitive core for robots. Our data can be found in https://mm-robobench.github.io/. △ Less

Submitted 28 June, 2024; originally announced June 2024.

Beyond First-Order: A Multi-Scale Approach to Finger Knuckle Print Biometrics

Authors: Chengrui Gao, Ziyuan Yang, Andrew Beng Jin Teoh, Min Zhu

Abstract: Recently, finger knuckle prints (FKPs) have gained attention due to their rich textural patterns, positioning them as a promising biometric for identity recognition. Prior FKP recognition methods predominantly leverage first-order feature descriptors, which capture intricate texture details but fail to account for structural information. Emerging research, however, indicates that second-order text… ▽ More Recently, finger knuckle prints (FKPs) have gained attention due to their rich textural patterns, positioning them as a promising biometric for identity recognition. Prior FKP recognition methods predominantly leverage first-order feature descriptors, which capture intricate texture details but fail to account for structural information. Emerging research, however, indicates that second-order textures, which describe the curves and arcs of the textures, encompass this overlooked structural information. This paper introduces a novel FKP recognition approach, the Dual-Order Texture Competition Network (DOTCNet), designed to capture texture information in FKP images comprehensively. DOTCNet incorporates three dual-order texture competitive modules (DTCMs), each targeting textures at different scales. Each DTCM employs a learnable texture descriptor, specifically a learnable Gabor filter (LGF), to extract texture features. By leveraging LGFs, the network extracts first and second order textures to describe fine textures and structural features thoroughly. Furthermore, an attention mechanism enhances relevant features in the first-order features, thereby highlighting significant texture details. For second-order features, a competitive mechanism emphasizes structural information while reducing noise from higher-order features. Extensive experimental results reveal that DOTCNet significantly outperforms several standard algorithms on the publicly available PolyU-FKP dataset. △ Less

Submitted 28 June, 2024; originally announced June 2024.

Self-consistent expansion and field-theoretic renormalization group for a singular nonlinear diffusion equation with anomalous scaling

Authors: Minhui Zhu, Nigel Goldenfeld

Abstract: The method of self-consistent expansions is a powerful tool for handling strong coupling problems that might otherwise be beyond the reach of perturbation theory, providing surprisingly accurate approximations even at low order. First applied in its embryonic form to fully-developed turbulence, it has subsequently been successfully applied to a variety of problems that include polymer statistics,… ▽ More The method of self-consistent expansions is a powerful tool for handling strong coupling problems that might otherwise be beyond the reach of perturbation theory, providing surprisingly accurate approximations even at low order. First applied in its embryonic form to fully-developed turbulence, it has subsequently been successfully applied to a variety of problems that include polymer statistics, interface dynamics and high order perturbation theory for the anharmonic oscillator. Here we show that the self-consistent expansion can be applied to singular perturbation problems arising in the theory of partial differential equations. We demonstrate its application to Barenblatt's nonlinear diffusion equation for porous media filtration, where the long-time asymptotics exhibits anomalous dimensions which can be systematically calculated using the perturbative renormalization group. We find that even the first order self-consistent expansion improves the approximation of the anomalous dimension obtained by the first order perturbative renormalization group, especially in the strong coupling regime. We also develop a field-theoretic framework for deterministic partial differential equations to facilitate the application of self-consistent expansions to other dynamic systems, and illustrate its application using the example of Barenblatt's equation. The scope of our results on the combination of renormalization group and self-consistent expansions is limited to partial differential equations whose long-time asymptotics is controlled by incomplete similarity. However, our work suggests that these methods could be applied to a broader suite of singular perturbation problems such as boundary layer theory, multiple scales analysis and matched asymptotic expansions, for which excellent approximations using renormalization group methods alone are already available. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Comments: 13 pages, 1 figure

Geometric Features Enhanced Human-Object Interaction Detection

Authors: Manli Zhu, Edmond S. L. Ho, Shuang Chen, Longzhi Yang, Hubert P. H. Shum

Abstract: Cameras are essential vision instruments to capture images for pattern detection and measurement. Human-object interaction (HOI) detection is one of the most popular pattern detection approaches for captured human-centric visual scenes. Recently, Transformer-based models have become the dominant approach for HOI detection due to their advanced network architectures and thus promising results. Howe… ▽ More Cameras are essential vision instruments to capture images for pattern detection and measurement. Human-object interaction (HOI) detection is one of the most popular pattern detection approaches for captured human-centric visual scenes. Recently, Transformer-based models have become the dominant approach for HOI detection due to their advanced network architectures and thus promising results. However, most of them follow the one-stage design of vanilla Transformer, leaving rich geometric priors under-exploited and leading to compromised performance especially when occlusion occurs. Given that geometric features tend to outperform visual ones in occluded scenarios and offer information that complements visual cues, we propose a novel end-to-end Transformer-style HOI detection model, i.e., geometric features enhanced HOI detector (GeoHOI). One key part of the model is a new unified self-supervised keypoint learning method named UniPointNet that bridges the gap of consistent keypoint representation across diverse object categories, including humans. GeoHOI effectively upgrades a Transformer-based HOI detector benefiting from the keypoints similarities measuring the likelihood of human-object interactions as well as local keypoint patches to enhance interaction query representation, so as to boost HOI predictions. Extensive experiments show that the proposed method outperforms the state-of-the-art models on V-COCO and achieves competitive performance on HICO-DET. Case study results on the post-disaster rescue with vision-based instruments showcase the applicability of the proposed GeoHOI in real-world applications. △ Less

Submitted 26 June, 2024; originally announced June 2024.

Comments: Accepted to IEEE TIM

APIGen: Automated Pipeline for Generating Verifiable and Diverse Function-Calling Datasets

Authors: Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, Rithesh Murthy, Liangwei Yang, Silvio Savarese, Juan Carlos Niebles, Huan Wang, Shelby Heinecke, Caiming Xiong

Abstract: The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets. This paper presents APIGen, an automated data generation pipeline designed to synthesize verifiable high-quality datasets for function-calling applications. We leverage APIGen and collect 3,673 executable APIs across 21 different categories to generate diverse function-calling datasets in a scal… ▽ More The advancement of function-calling agent models requires diverse, reliable, and high-quality datasets. This paper presents APIGen, an automated data generation pipeline designed to synthesize verifiable high-quality datasets for function-calling applications. We leverage APIGen and collect 3,673 executable APIs across 21 different categories to generate diverse function-calling datasets in a scalable and structured manner. Each data in our dataset is verified through three hierarchical stages: format checking, actual function executions, and semantic verification, ensuring its reliability and correctness. We demonstrate that models trained with our curated datasets, even with only 7B parameters, can achieve state-of-the-art performance on the Berkeley Function-Calling Benchmark, outperforming multiple GPT-4 models. Moreover, our 1B model achieves exceptional performance, surpassing GPT-3.5-Turbo and Claude-3 Haiku. We release a dataset containing 60,000 high-quality entries, aiming to advance the field of function-calling agent domains. The dataset is available on Huggingface: https://huggingface.co/datasets/Salesforce/xlam-function-calling-60k and the project homepage: https://apigen-pipeline.github.io/ △ Less

Submitted 26 June, 2024; originally announced June 2024.

Dependence Analysis and Structured Construction for Batched Sparse Code

Authors: Jiaxin Qing, Xiaohong Cai, Yijun Fan, Mingyang Zhu, Raymond W. Yeung

Abstract: In coding theory, codes are usually designed with a certain level of randomness to facilitate analysis and accommodate different channel conditions. However, the resulting random code constructed can be suboptimal in practical implementations. Represented by a bipartite graph, the Batched Sparse Code (BATS Code) is a randomly constructed erasure code that utilizes network coding to achieve near-op… ▽ More In coding theory, codes are usually designed with a certain level of randomness to facilitate analysis and accommodate different channel conditions. However, the resulting random code constructed can be suboptimal in practical implementations. Represented by a bipartite graph, the Batched Sparse Code (BATS Code) is a randomly constructed erasure code that utilizes network coding to achieve near-optimal performance in wireless multi-hop networks. In the performance analysis in the previous research, it is implicitly assumed that the coded batches in the BATS code are independent. This assumption holds only asymptotically when the number of input symbols is infinite, but it does not generally hold in a practical setting where the number of input symbols is finite, especially when the code is constructed randomly. We show that dependence among the batches significantly degrades the code's performance. In order to control the batch dependence through graphical design, we propose constructing the BATS code in a structured manner. A hardware-friendly structured BATS code called the Cyclic-Shift BATS (CS-BATS) code is proposed, which constructs the code from a small base graph using light-weight cyclic-shift operations. We demonstrate that when the base graph is properly designed, a higher decoding rate and a smaller complexity can be achieved compared with the random BATS code. △ Less

Submitted 25 June, 2024; originally announced June 2024.

MetaFollower: Adaptable Personalized Autonomous Car Following

Authors: Xianda Chen, Kehua Chen, Meixin Zhu, Hao, Yang, Shaojie Shen, Xuesong Wang, Yinhai Wang

Abstract: Car-following (CF) modeling, a fundamental component in microscopic traffic simulation, has attracted increasing interest of researchers in the past decades. In this study, we propose an adaptable personalized car-following framework -MetaFollower, by leveraging the power of meta-learning. Specifically, we first utilize Model-Agnostic Meta-Learning (MAML) to extract common driving knowledge from v… ▽ More Car-following (CF) modeling, a fundamental component in microscopic traffic simulation, has attracted increasing interest of researchers in the past decades. In this study, we propose an adaptable personalized car-following framework -MetaFollower, by leveraging the power of meta-learning. Specifically, we first utilize Model-Agnostic Meta-Learning (MAML) to extract common driving knowledge from various CF events. Afterward, the pre-trained model can be fine-tuned on new drivers with only a few CF trajectories to achieve personalized CF adaptation. We additionally combine Long Short-Term Memory (LSTM) and Intelligent Driver Model (IDM) to reflect temporal heterogeneity with high interpretability. Unlike conventional adaptive cruise control (ACC) systems that rely on predefined settings and constant parameters without considering heterogeneous driving characteristics, MetaFollower can accurately capture and simulate the intricate dynamics of car-following behavior while considering the unique driving styles of individual drivers. We demonstrate the versatility and adaptability of MetaFollower by showcasing its ability to adapt to new drivers with limited training data quickly. To evaluate the performance of MetaFollower, we conduct rigorous experiments comparing it with both data-driven and physics-based models. The results reveal that our proposed framework outperforms baseline models in predicting car-following behavior with higher accuracy and safety. To the best of our knowledge, this is the first car-following model aiming to achieve fast adaptation by considering both driver and temporal heterogeneity based on meta-learning. △ Less

Submitted 23 June, 2024; originally announced June 2024.