Showing 1–50 of 501 results for author: Xiong, J

Zero-sum stochastic linear-quadratic Stackelberg differential games of Markovian regime-switching system

Authors: Fan Wu, Xun Li, Jie Xiong, Xin Zhang

Abstract: This paper investigates a zero-sum stochastic linear-quadratic (SLQ, for short) Stackelberg differential game problem, where the coefficients of the state equation and the weighting matrices in the performance functional are regulated by a Markov chain. By utilizing the findings in \citet{Zhang.X.2021_ILQM}, we directly present the feedback representation to the rational reaction of the follower.… ▽ More This paper investigates a zero-sum stochastic linear-quadratic (SLQ, for short) Stackelberg differential game problem, where the coefficients of the state equation and the weighting matrices in the performance functional are regulated by a Markov chain. By utilizing the findings in \citet{Zhang.X.2021_ILQM}, we directly present the feedback representation to the rational reaction of the follower. For the leader's problem, we derive the optimality system through the variational method and study its unique solvability from the Hilbert space point of view. We construct the explicit optimal control for the leader based on the solution to coupled differential Riccati equations (CDREs, for short) and obtain the solvability of CDREs under the one-dimensional framework. Finally, we provide two concrete examples to illustrate the results developed in this paper. △ Less

Submitted 30 August, 2024; originally announced August 2024.

MSC Class: 91A15; 49N10; 93E20

Adiabatic Mass Loss in Binary Stars. V. Effects of Metallicity and Nonconservative Mass Transfer -- Application in High Mass X-ray Binaries

Authors: Hongwei Ge, Christopher Adam Tout, Xuefei Chen, Song Wang, Jianping Xiong, Lifu Zhang, Qingzhong Liu, Zhanwen Han

Abstract: Binary stars are responsible for many unusual astrophysical phenomena, including some important explosive cosmic events. The stability criteria for rapid mass transfer and common-envelope evolution are fundamental to binary star evolution. They determine the mass, mass ratio, and orbital distribution of systems such as X-ray binaries and merging gravitational-wave sources. We use our adiabatic mas… ▽ More Binary stars are responsible for many unusual astrophysical phenomena, including some important explosive cosmic events. The stability criteria for rapid mass transfer and common-envelope evolution are fundamental to binary star evolution. They determine the mass, mass ratio, and orbital distribution of systems such as X-ray binaries and merging gravitational-wave sources. We use our adiabatic mass-loss model to systematically survey metal-poor and solar-metallicity donor thresholds for dynamical timescale mass transfer. The critical mass ratios qad are systematically explored, and the impact of metallicity and nonconservative mass transfer are studied. For metal-poor radiative-envelope donors, qad are smaller than those for solar-metallicity stars at the same evolutionary stage. However, qad do the opposite for convective-envelope donors. Nonconservative mass transfer significantly decreases qad for massive donors. This is because it matters how conservative mass transfer is during the thermal timescale phase immediately preceding a delayed dynamical mass transfer. We apply our theoretical predictions to observed high-mass X-ray binaries that have overfilled their Roche lobes and find a good agreement with their mass ratios. Our results can be applied to study individual binary objects or large samples of binary objects with binary population synthesis codes. △ Less

Submitted 29 August, 2024; originally announced August 2024.

Comments: Submitted to ApJ. Comments are welcome

Stochastic linear-quadratic differential game with Markovian jumps in an infinite horizon

Authors: Fan Wu, Xun Li, Jie Xiong, Xin Zhang

Abstract: This paper investigates a two-person non-homogeneous linear-quadratic stochastic differential game (LQ-SDG, for short) in an infinite horizon for a system regulated by a time-invariant Markov chain. Both non-zero-sum and zero-sum LQ-SDG problems are studied. It is shown that the zero-sum LQ-SDG problem can be considered a special non-zero-sum LQ-SDG problem. The open-loop Nash equilibrium point of… ▽ More This paper investigates a two-person non-homogeneous linear-quadratic stochastic differential game (LQ-SDG, for short) in an infinite horizon for a system regulated by a time-invariant Markov chain. Both non-zero-sum and zero-sum LQ-SDG problems are studied. It is shown that the zero-sum LQ-SDG problem can be considered a special non-zero-sum LQ-SDG problem. The open-loop Nash equilibrium point of non-zero-sum (zero-sum, respectively) LQ-SDG problem is characterized by the solvability of a system of constrained forward-backward stochastic differential equations (FBSDEs, for short) in an infinite horizon and the convexity (convexity-concavity, respectively) of the performance functional and their corresponding closed-loop Nash equilibrium strategy are characterized by the solvability of a system of constrained coupled algebra Riccati equations (CAREs, for short) with certain stabilizing conditions. In addition, the closed-loop representation of open-loop Nash equilibrium point for non-zero-sum (zero-sum, respectively) LQ-SDG is provided by the non-symmetric (symmetric, respectively) solution to a system CAREs. At the end of this paper, we provide three concrete examples and solve their open-loop/closed-loop Nash equilibrium strategy based on the obtained results. △ Less

Submitted 22 August, 2024; originally announced August 2024.

MSC Class: 93E03; 93E20

Estimating the Atmospheric Parameters of Early-type Stars from the Chinese Space Station Telescope (CSST) Slitless Spectra Survey

Authors: JiaRui Rao, HaiLiang Chen, JianPing Xiong, LuQian Wang, YanJun Guo, JiaJia Li, Chao Liu, ZhanWen Han, XueFei Chen

Abstract: The measurement of atmospheric parameters is fundamental for scientific research using stellar spectra. The Chinese Space Station Telescope (CSST), scheduled to be launched in 2024, will provide researchers with hundreds of millions of slitless spectra for stars during a 10 yr survey. And machine learning has unparalleled efficiency in processing large amounts of data compared to manual processing… ▽ More The measurement of atmospheric parameters is fundamental for scientific research using stellar spectra. The Chinese Space Station Telescope (CSST), scheduled to be launched in 2024, will provide researchers with hundreds of millions of slitless spectra for stars during a 10 yr survey. And machine learning has unparalleled efficiency in processing large amounts of data compared to manual processing. Here we studied the stellar parameters of early-type stars (effective temperature Teff more than 15,000 K) based on the design indicators of the CSST slitless spectrum and the machine learning algorithm, Stellar LAbel Machine. We used the Potsdam Wolf-Rayet (POWR) synthetic spectra library for cross validation. Then we tested the reliability of machine learning results by using the Next Generation Spectrum Library (NGSL) from Hubble Space Telescope observation data. We use the spectra with the impact of interstellar extinction (AV = 0, 0.5, 1, 1.5, 2 mag) and radial velocity (RV = -50, -30, 0, 30, 50 km s-1) from the POWR library as the test set. When RV = 0 km s-1 and AV = 0 mag, the average value and standard deviation for 3 wavelength ranges (2550-4050 Ang (R = 287); 4050-6300 Ang (R = 232); 6300-10000 Ang (R = 207)) are -66 K, 550 K, and 356 K for Teff, and 0.004 c.g.s, -0.024 c.g.s, and 0.01 c.g.s for log g. When using the observed data from NGSL as the testing samples, the deviation of Teff is less than 5%, and the deviation of log g is less than 11%. In addition, we also test the influence of shifting of spectra on the parameters accuracy. The deviation of Teff for the case with a shift of 5 Ang and 10 Ang are 3.6% and 4.3%, respectively; the deviation of log g are 4.2% and 5.1%. These results demonstrate that we can obtain relatively accurate stellar parameters of a population of early-type stars with the CSST slitless spectra and a machine-learning method. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Journal ref: The Astronomical Journal, 168:20 (17pp), 2024 July

LLaVA-Surg: Towards Multimodal Surgical Assistant via Structured Surgical Video Learning

Authors: Jiajie Li, Garrett Skinner, Gene Yang, Brian R Quaranto, Steven D Schwaitzberg, Peter C W Kim, Jinjun Xiong

Abstract: Multimodal large language models (LLMs) have achieved notable success across various domains, while research in the medical field has largely focused on unimodal images. Meanwhile, current general-domain multimodal models for videos still lack the capabilities to understand and engage in conversations about surgical videos. One major contributing factor is the absence of datasets in the surgical f… ▽ More Multimodal large language models (LLMs) have achieved notable success across various domains, while research in the medical field has largely focused on unimodal images. Meanwhile, current general-domain multimodal models for videos still lack the capabilities to understand and engage in conversations about surgical videos. One major contributing factor is the absence of datasets in the surgical field. In this paper, we create a new dataset, Surg-QA, consisting of 102,000 surgical video-instruction pairs, the largest of its kind so far. To build such a dataset, we propose a novel two-stage question-answer generation pipeline with LLM to learn surgical knowledge in a structured manner from the publicly available surgical lecture videos. The pipeline breaks down the generation process into two stages to significantly reduce the task complexity, allowing us to use a more affordable, locally deployed open-source LLM than the premium paid LLM services. It also mitigates the risk of LLM hallucinations during question-answer generation, thereby enhancing the overall quality of the generated data. We further train LLaVA-Surg, a novel vision-language conversational assistant capable of answering open-ended questions about surgical videos, on this Surg-QA dataset, and conduct comprehensive evaluations on zero-shot surgical video question-answering tasks. We show that LLaVA-Surg significantly outperforms all previous general-domain models, demonstrating exceptional multimodal conversational skills in answering open-ended questions about surgical videos. We will release our code, model, and the instruction-tuning dataset. △ Less

Submitted 15 August, 2024; originally announced August 2024.

Assessment of Cell Nuclei AI Foundation Models in Kidney Pathology

Authors: Junlin Guo, Siqi Lu, Can Cui, Ruining Deng, Tianyuan Yao, Zhewen Tao, Yizhe Lin, Marilyn Lionts, Quan Liu, Juming Xiong, Catie Chang, Mitchell Wilkes, Mengmeng Yin, Haichun Yang, Yuankai Huo

Abstract: Cell nuclei instance segmentation is a crucial task in digital kidney pathology. Traditional automatic segmentation methods often lack generalizability when applied to unseen datasets. Recently, the success of foundation models (FMs) has provided a more generalizable solution, potentially enabling the segmentation of any cell type. In this study, we perform a large-scale evaluation of three widely… ▽ More Cell nuclei instance segmentation is a crucial task in digital kidney pathology. Traditional automatic segmentation methods often lack generalizability when applied to unseen datasets. Recently, the success of foundation models (FMs) has provided a more generalizable solution, potentially enabling the segmentation of any cell type. In this study, we perform a large-scale evaluation of three widely used state-of-the-art (SOTA) cell nuclei foundation models (Cellpose, StarDist, and CellViT). Specifically, we created a highly diverse evaluation dataset consisting of 2,542 kidney whole slide images (WSIs) collected from both human and rodent sources, encompassing various tissue types, sizes, and staining methods. To our knowledge, this is the largest-scale evaluation of its kind to date. Our quantitative analysis of the prediction distribution reveals a persistent performance gap in kidney pathology. Among the evaluated models, CellViT demonstrated superior performance in segmenting nuclei in kidney pathology. However, none of the foundation models are perfect; a performance gap remains in general nuclei segmentation for kidney pathology. △ Less

Submitted 9 August, 2024; originally announced August 2024.

Drone based superconducting single photon detection system with detection efficiency more than 90%

Authors: Ruoyan Ma, Zhimin Guo, Dai Chen, Xiaojun Dai, You Xiao, ChengJun Zhang, Jiamin Xiong, Jia Huang, Xingyu Zhang, Xiaoyu Liu, Liangliang Rong, Hao Li, Xiaofu Zhang, Lixing You

Abstract: Bounded by the size, weight, and power consumption (SWaP) of conventional superconducting single photon detectors (SSPD), applications of SSPDs were commonly confined in the laboratory. However, booming demands for high efficiency single photon detector incorporated with avionic platforms arise with the development of remote imaging and sensing or long-haul quantum communication without topographi… ▽ More Bounded by the size, weight, and power consumption (SWaP) of conventional superconducting single photon detectors (SSPD), applications of SSPDs were commonly confined in the laboratory. However, booming demands for high efficiency single photon detector incorporated with avionic platforms arise with the development of remote imaging and sensing or long-haul quantum communication without topographical constraints. We herein designed and manufactured the first drone based SSPD system with a SDE as high as 91.8%. This drone based SSPD system is established with high performance NbTiN SSPDs, self-developed miniature liquid helium dewar, and homemade integrated electric setups, which is able to be launched in complex topographical conditions. Such a drone based SSPD system may open the use of SSPDs for applications that demand high-SDE in complex environments. △ Less

Submitted 11 August, 2024; originally announced August 2024.

Optimal Frequency in Second Messenger Signaling Quantifying cAMP Information Transmission in Bacteria

Authors: Jiarui Xiong, Liang Wang, Jialun Lin, Lei Ni, Rongrong Zhang, Shuai Yang, Yajia Huang, Jun Chu, Fan Jin

Abstract: Bacterial second messengers are crucial for transmitting environmental information to cellular responses. However, quantifying their information transmission capacity remains challenging. Here, we engineer an isolated cAMP signaling channel in Pseudomonas aeruginosa using targeted gene knockouts, optogenetics, and a fluorescent cAMP probe. This design allows precise optical control and real-time m… ▽ More Bacterial second messengers are crucial for transmitting environmental information to cellular responses. However, quantifying their information transmission capacity remains challenging. Here, we engineer an isolated cAMP signaling channel in Pseudomonas aeruginosa using targeted gene knockouts, optogenetics, and a fluorescent cAMP probe. This design allows precise optical control and real-time monitoring of cAMP dynamics. By integrating experimental data with information theory, we reveal an optimal frequency for light-mediated cAMP signaling that maximizes information transmission, reaching about 40 bits/h. This rate correlates strongly with cAMP degradation kinetics and employs a two-state encoding scheme. Our findings suggest a mechanism for fine-tuned regulation of multiple genes through temporal encoding of second messenger signals, providing new insights into bacterial adaptation strategies. This approach offers a framework for quantifying information processing in cellular signaling systems. △ Less

Submitted 9 August, 2024; originally announced August 2024.

Comments: 33 pages, 4 figures

MSC Class: 92-05; 92-10 ACM Class: J.2.4

A Method of Rapidly Deriving Late-type Contact Binary Parameters and Its Application in the Catalina Sky Survey

Authors: JinLiang Wang, Xu Ding, JiaJia Li, JianPing Xiong, Qiyuan Cheng, KaiFan Ji

Abstract: With the continuous development of large optical surveys, a large number of light curves of late-type contact binary systems (CBs) have been released. Deriving parameters for CBs using the the WD program and the PHOEBE program poses a challenge. Therefore, this study developed a method for rapidly deriving light curves based on the Neural Networks (NN) model combined with the Hamiltonian Monte Car… ▽ More With the continuous development of large optical surveys, a large number of light curves of late-type contact binary systems (CBs) have been released. Deriving parameters for CBs using the the WD program and the PHOEBE program poses a challenge. Therefore, this study developed a method for rapidly deriving light curves based on the Neural Networks (NN) model combined with the Hamiltonian Monte Carlo (HMC) algorithm (NNHMC). The neural network was employed to establish the mapping relationship between the parameters and the pregenerated light curves by the PHOEBE program, and the HMC algorithm was used to obtain the posterior distribution of the parameters. The NNHMC method was applied to a large contact binary sample from the Catalina Sky Survey, and a total of 19,104 late-type contact binary parameters were derived. Among them, 5172 have an inclination greater than 70 deg and a temperature difference less than 400 K. The obtained results were compared with the previous studies for 30 CBs, and there was an essentially consistent goodness-of-fit (R2) distribution between them. The NNHMC method possesses the capability to simultaneously derive parameters for a vast number of targets. Furthermore, it can provide an extremely efficient tool for rapid derivation of parameters in future sky surveys involving large samples of CBs. △ Less

Submitted 9 August, 2024; originally announced August 2024.

Journal ref: The Astrophysical Journal Supplement Series. Published 2024 July 31

Energy-lowering symmetry breaking creates a flat-band insulator in paramagnetic Nb3Cl8

Authors: Jia-Xin Xiong, Xiuwen Zhang, Alex Zunger

Abstract: Ordinary band structure calculations of quantum materials often incorrectly predicted metallic, instead of insulating electronic structure, motivating Mott-Hubbard strong electron correlation as a gapping mechanism. More recently, allowing the formation of local structural symmetry breaking motifs in otherwise ordinary band theory was shown to lower the total energy while predicting insulating gap… ▽ More Ordinary band structure calculations of quantum materials often incorrectly predicted metallic, instead of insulating electronic structure, motivating Mott-Hubbard strong electron correlation as a gapping mechanism. More recently, allowing the formation of local structural symmetry breaking motifs in otherwise ordinary band theory was shown to lower the total energy while predicting insulating gaps when they are observed. An important counter example was recently pointed out whereby the flat band formed in Nb3Cl8 by symmetry breaking is by itself partially occupied, thus failing to account for the observed insulating state. It is shown here that a generalized symmetry breaking involving the cooperative structural and magnetic effects produces in mean-field-like density functional theory an energy lowering insulating phase. △ Less

Submitted 31 July, 2024; originally announced August 2024.

Comments: 3 figures

Exploring quantum sensing for fine-grained liquid recognition

Authors: Yuechun Jiao, Jinlian Hu, Zitong Lan, Fusang Zhang, Jie Xiong, Jingxu Bai, Zhaoxin Chang, Yuqi Su, Beihong Jin, Daqing Zhang, Jianming Zhao, Suotang Jia

Abstract: Recent years have witnessed the use of pervasive wireless signals (e.g., Wi-Fi, RFID, and mmWave) for sensing purposes. Due to its non-intrusive characteristic, wireless sensing plays an important role in various intelligent sensing applications. However, limited by the inherent thermal noise of RF transceivers, the sensing granularity of existing wireless sensing systems are still coarse, limitin… ▽ More Recent years have witnessed the use of pervasive wireless signals (e.g., Wi-Fi, RFID, and mmWave) for sensing purposes. Due to its non-intrusive characteristic, wireless sensing plays an important role in various intelligent sensing applications. However, limited by the inherent thermal noise of RF transceivers, the sensing granularity of existing wireless sensing systems are still coarse, limiting their adoption for fine-grained sensing applications. In this paper, we introduce the quantum receiver, which does not contain traditional electronic components such as mixers, amplifiers, and analog-to-digital converters (ADCs) to wireless sensing systems, significantly reducing the source of thermal noise. By taking non-intrusive liquid recognition as an application example, we show the superior performance of quantum wireless sensing. By leveraging the unique property of quantum receiver, we propose a novel double-ratio method to address several well-known challenges in liquid recognition, eliminating the effect of liquid volume, device-target distance and container. We implement the quantum sensing prototype and evaluate the liquid recognition performance comprehensively. The results show that our system is able to recognize 17 commonly seen liquids, including very similar ones~(e.g., Pepsi and Coke) at an accuracy higher than 99.9\%. For milk expiration monitoring, our system is able to achieve an accuracy of 99.0\% for pH value measurements at a granularity of 0.1, which is much finer than that required for expiration monitoring. △ Less

Submitted 28 July, 2024; originally announced July 2024.

Comments: 7 pages, 6 figures

Quantitative diffusion approximation for the Neutral $r$-Alleles Wright-Fisher Model with Mutations

Authors: Peng Chen, Jie Xiong, Lihu Xu, Jiayu Zheng

Abstract: We apply a Lindeberg principle under the Markov process setting to approximate the Wright-Fisher model with neutral $r$-alleles using a diffusion process, deriving an error rate based on a function class distance involving fourth-order bounded differentiable functions. This error rate consists of a linear combination of the maximum mutation rate and the reciprocal of the population size. Our resul… ▽ More We apply a Lindeberg principle under the Markov process setting to approximate the Wright-Fisher model with neutral $r$-alleles using a diffusion process, deriving an error rate based on a function class distance involving fourth-order bounded differentiable functions. This error rate consists of a linear combination of the maximum mutation rate and the reciprocal of the population size. Our result improves the error bound in the seminal work [PNAS,1977], where only the special case $r=2$ was studied. △ Less

Submitted 12 July, 2024; originally announced July 2024.

SpreadsheetLLM: Encoding Spreadsheets for Large Language Models

Authors: Yuzhang Tian, Jianbo Zhao, Haoyu Dong, Junyu Xiong, Shiyu Xia, Mengyu Zhou, Yun Lin, José Cambronero, Yeye He, Shi Han, Dongmei Zhang

Abstract: Spreadsheets, with their extensive two-dimensional grids, various layouts, and diverse formatting options, present notable challenges for large language models (LLMs). In response, we introduce SpreadsheetLLM, pioneering an efficient encoding method designed to unleash and optimize LLMs' powerful understanding and reasoning capability on spreadsheets. Initially, we propose a vanilla serialization… ▽ More Spreadsheets, with their extensive two-dimensional grids, various layouts, and diverse formatting options, present notable challenges for large language models (LLMs). In response, we introduce SpreadsheetLLM, pioneering an efficient encoding method designed to unleash and optimize LLMs' powerful understanding and reasoning capability on spreadsheets. Initially, we propose a vanilla serialization approach that incorporates cell addresses, values, and formats. However, this approach was limited by LLMs' token constraints, making it impractical for most applications. To tackle this challenge, we develop SheetCompressor, an innovative encoding framework that compresses spreadsheets effectively for LLMs. It comprises three modules: structural-anchor-based compression, inverse index translation, and data-format-aware aggregation. It significantly improves performance in spreadsheet table detection task, outperforming the vanilla approach by 25.6% in GPT4's in-context learning setting. Moreover, fine-tuned LLM with SheetCompressor has an average compression ratio of 25 times, but achieves a state-of-the-art 78.9% F1 score, surpassing the best existing models by 12.3%. Finally, we propose Chain of Spreadsheet for downstream tasks of spreadsheet understanding and validate in a new and demanding spreadsheet QA task. We methodically leverage the inherent layout and structure of spreadsheets, demonstrating that SpreadsheetLLM is highly effective across a variety of spreadsheet tasks. △ Less

Submitted 12 July, 2024; originally announced July 2024.

Lite-SAM Is Actually What You Need for Segment Everything

Authors: Jianhai Fu, Yuanjie Yu, Ningchuan Li, Yi Zhang, Qichao Chen, Jianping Xiong, Jun Yin, Zhiyu Xiang

Abstract: This paper introduces Lite-SAM, an efficient end-to-end solution for the SegEvery task designed to reduce computational costs and redundancy. Lite-SAM is composed of four main components: a streamlined CNN-Transformer hybrid encoder (LiteViT), an automated prompt proposal network (AutoPPN), a traditional prompt encoder, and a mask decoder. All these components are integrated within the SAM framewo… ▽ More This paper introduces Lite-SAM, an efficient end-to-end solution for the SegEvery task designed to reduce computational costs and redundancy. Lite-SAM is composed of four main components: a streamlined CNN-Transformer hybrid encoder (LiteViT), an automated prompt proposal network (AutoPPN), a traditional prompt encoder, and a mask decoder. All these components are integrated within the SAM framework. Our LiteViT, a high-performance lightweight backbone network, has only 1.16M parameters, which is a 23% reduction compared to the lightest existing backbone network Shufflenet. We also introduce AutoPPN, an innovative end-to-end method for prompt boxes and points generation. This is an improvement over traditional grid search sampling methods, and its unique design allows for easy integration into any SAM series algorithm, extending its usability. we have thoroughly benchmarked Lite-SAM across a plethora of both public and private datasets. The evaluation encompassed a broad spectrum of universal metrics, including the number of parameters, SegEvery execution time, and accuracy. The findings reveal that Lite-SAM, operating with a lean 4.2M parameters, significantly outpaces its counterparts, demonstrating performance improvements of 43x, 31x, 20x, 21x, and 1.6x over SAM, MobileSAM, Edge-SAM, EfficientViT-SAM, and MobileSAM-v2 respectively, all the while maintaining competitive accuracy. This underscores Lite-SAM's prowess in achieving an optimal equilibrium between performance and precision, thereby setting a new state-of-the-art(SOTA) benchmark in the domain. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: ECCV 2024 Accepted

Extinction profiles for the Sobolev critical fast diffusion equation in bounded domains. I. One bubble dynamics

Authors: Tianling Jin, Jingang Xiong

Abstract: In this paper, we investigate the extinction behavior of nonnegative solutions to the Sobolev critical fast diffusion equation in bounded smooth domains with the Dirichlet zero boundary condition. Under the two-bubble energy threshold assumption on the initial data, we prove the dichotomy that every solution converges uniformly, in terms of relative error, to either a steady state or a blowing-up… ▽ More In this paper, we investigate the extinction behavior of nonnegative solutions to the Sobolev critical fast diffusion equation in bounded smooth domains with the Dirichlet zero boundary condition. Under the two-bubble energy threshold assumption on the initial data, we prove the dichotomy that every solution converges uniformly, in terms of relative error, to either a steady state or a blowing-up bubble. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Comments: 40 pages, comments are welcome

Experimental Demonstration of 16D Voronoi Constellation with Two-Level Coding over 50km Four-Core Fiber

Authors: Can Zhao, Bin Chen, Jiaqi Cai, Zhiwei Liang, Yi Lei, Junjie Xiong, Lin Ma, Daohui Hu, Lin Sun, Gangxiang Shen

Abstract: A 16-dimensional Voronoi constellation concatenated with multilevel coding is experimentally demonstrated over a 50km four-core fiber transmission system. The proposed scheme reduces the required launch power by 6dB and provides a 17dB larger operating range than 16QAM with BICM at the outer HD-FEC BER threshold. A 16-dimensional Voronoi constellation concatenated with multilevel coding is experimentally demonstrated over a 50km four-core fiber transmission system. The proposed scheme reduces the required launch power by 6dB and provides a 17dB larger operating range than 16QAM with BICM at the outer HD-FEC BER threshold. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Comments: 4 pages, 4 figures, accepted by 2024 European Conference on Optical Communication (ECOC)

Shared-Protected Backup Paths Assignment with Mode Group Division Multiplexing in Optical Networks

Authors: Jiaheng Xiong, Qiaolun Zhang, Ruikun Wang, Alberto Gatto, Francesco Musumeci, Massimo Tornatore

Abstract: We evaluate the resource efficiency of Mode Group Division Multiplexing (MGDM) with shared path protection (SPP) in optical networks. On our case studies, SPP with MGDM obtains significant savings in terms of both additional backup spectrum occupation and MIMO-computing resources compared to other few-mode-transmission scenarios. We evaluate the resource efficiency of Mode Group Division Multiplexing (MGDM) with shared path protection (SPP) in optical networks. On our case studies, SPP with MGDM obtains significant savings in terms of both additional backup spectrum occupation and MIMO-computing resources compared to other few-mode-transmission scenarios. △ Less

Submitted 2 July, 2024; originally announced July 2024.

HATs: Hierarchical Adaptive Taxonomy Segmentation for Panoramic Pathology Image Analysis

Authors: Ruining Deng, Quan Liu, Can Cui, Tianyuan Yao, Juming Xiong, Shunxing Bao, Hao Li, Mengmeng Yin, Yu Wang, Shilin Zhao, Yucheng Tang, Haichun Yang, Yuankai Huo

Abstract: Panoramic image segmentation in computational pathology presents a remarkable challenge due to the morphologically complex and variably scaled anatomy. For instance, the intricate organization in kidney pathology spans multiple layers, from regions like the cortex and medulla to functional units such as glomeruli, tubules, and vessels, down to various cell types. In this paper, we propose a novel… ▽ More Panoramic image segmentation in computational pathology presents a remarkable challenge due to the morphologically complex and variably scaled anatomy. For instance, the intricate organization in kidney pathology spans multiple layers, from regions like the cortex and medulla to functional units such as glomeruli, tubules, and vessels, down to various cell types. In this paper, we propose a novel Hierarchical Adaptive Taxonomy Segmentation (HATs) method, which is designed to thoroughly segment panoramic views of kidney structures by leveraging detailed anatomical insights. Our approach entails (1) the innovative HATs technique which translates spatial relationships among 15 distinct object classes into a versatile "plug-and-play" loss function that spans across regions, functional units, and cells, (2) the incorporation of anatomical hierarchies and scale considerations into a unified simple matrix representation for all panoramic entities, (3) the adoption of the latest AI foundation model (EfficientSAM) as a feature extraction tool to boost the model's adaptability, yet eliminating the need for manual prompt generation in conventional segment anything model (SAM). Experimental findings demonstrate that the HATs method offers an efficient and effective strategy for integrating clinical insights and imaging precedents into a unified segmentation model across more than 15 categories. The official implementation is publicly available at https://github.com/hrlblab/HATs. △ Less

Submitted 30 June, 2024; originally announced July 2024.

Comments: arXiv admin note: text overlap with arXiv:2402.19286

Weighted Circle Fusion: Ensembling Circle Representation from Different Object Detection Results

Authors: Jialin Yue, Tianyuan Yao, Ruining Deng, Quan Liu, Juming Xiong, Haichun Yang, Yuankai Huo

Abstract: Recently, the use of circle representation has emerged as a method to improve the identification of spherical objects (such as glomeruli, cells, and nuclei) in medical imaging studies. In traditional bounding box-based object detection, combining results from multiple models improves accuracy, especially when real-time processing isn't crucial. Unfortunately, this widely adopted strategy is not re… ▽ More Recently, the use of circle representation has emerged as a method to improve the identification of spherical objects (such as glomeruli, cells, and nuclei) in medical imaging studies. In traditional bounding box-based object detection, combining results from multiple models improves accuracy, especially when real-time processing isn't crucial. Unfortunately, this widely adopted strategy is not readily available for combining circle representations. In this paper, we propose Weighted Circle Fusion (WCF), a simple approach for merging predictions from various circle detection models. Our method leverages confidence scores associated with each proposed bounding circle to generate averaged circles. Our method undergoes thorough evaluation on a proprietary dataset for glomerular detection in object detection within whole slide imaging (WSI). The findings reveal a performance gain of 5 %, respectively, compared to existing ensemble methods. Furthermore, the Weighted Circle Fusion technique not only improves the precision of object detection in medical images but also notably decreases false detections, presenting a promising direction for future research and application in pathological image analysis. △ Less

Submitted 27 June, 2024; originally announced June 2024.

FASA: a Flexible and Automatic Speech Aligner for Extracting High-quality Aligned Children Speech Data

Authors: Dancheng Liu, Jinjun Xiong

Abstract: Automatic Speech Recognition (ASR) for adults' speeches has made significant progress by employing deep neural network (DNN) models recently, but improvement in children's speech is still unsatisfactory due to children's speech's distinct characteristics. DNN models pre-trained on adult data often struggle in generalizing children's speeches with fine tuning because of the lack of high-quality ali… ▽ More Automatic Speech Recognition (ASR) for adults' speeches has made significant progress by employing deep neural network (DNN) models recently, but improvement in children's speech is still unsatisfactory due to children's speech's distinct characteristics. DNN models pre-trained on adult data often struggle in generalizing children's speeches with fine tuning because of the lack of high-quality aligned children's speeches. When generating datasets, human annotations are not scalable, and existing forced-alignment tools are not usable as they make impractical assumptions about the quality of the input transcriptions. To address these challenges, we propose a new forced-alignment tool, FASA, as a flexible and automatic speech aligner to extract high-quality aligned children's speech data from many of the existing noisy children's speech data. We demonstrate its usage on the CHILDES dataset and show that FASA can improve data quality by 13.6$\times$ over human annotations. △ Less

Submitted 25 June, 2024; originally announced June 2024.

Comments: 4 pages, 1 figure

Large Language Models have Intrinsic Self-Correction Ability

Authors: Dancheng Liu, Amir Nassereldine, Ziming Yang, Chenhui Xu, Yuting Hu, Jiajie Li, Utkarsh Kumar, Changjae Lee, Jinjun Xiong

Abstract: Large language models (LLMs) have attracted significant attention for their remarkable abilities in various natural language processing tasks, but they suffer from hallucinations that will cause performance degradation. One promising solution to improve the LLMs' performance is to ask LLMs to revise their answer after generation, a technique known as self-correction. Among the two types of self-co… ▽ More Large language models (LLMs) have attracted significant attention for their remarkable abilities in various natural language processing tasks, but they suffer from hallucinations that will cause performance degradation. One promising solution to improve the LLMs' performance is to ask LLMs to revise their answer after generation, a technique known as self-correction. Among the two types of self-correction, intrinsic self-correction is considered a promising direction because it does not utilize external knowledge. However, recent works doubt the validity of LLM's ability to conduct intrinsic self-correction. In this paper, we present a novel perspective on the intrinsic self-correction capabilities of LLMs through theoretical analyses and empirical experiments. In addition, we identify two critical factors for successful self-correction: zero temperature and fair prompts. Leveraging these factors, we demonstrate that intrinsic self-correction ability is exhibited across multiple existing LLMs. Our findings offer insights into the fundamental theories underlying the self-correction behavior of LLMs and remark on the importance of unbiased prompts and zero temperature settings in harnessing their full potential. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Comments: in submission

PI-Whisper: An Adaptive and Incremental ASR Framework for Diverse and Evolving Speaker Characteristics

Authors: Amir Nassereldine, Dancheng Liu, Chenhui Xu, Jinjun Xiong

Abstract: As edge-based automatic speech recognition (ASR) technologies become increasingly prevalent for the development of intelligent and personalized assistants, three important challenges must be addressed for these resource-constrained ASR models, i.e., adaptivity, incrementality, and inclusivity. We propose a novel ASR framework, PI-Whisper, in this work and show how it can improve an ASR's recogniti… ▽ More As edge-based automatic speech recognition (ASR) technologies become increasingly prevalent for the development of intelligent and personalized assistants, three important challenges must be addressed for these resource-constrained ASR models, i.e., adaptivity, incrementality, and inclusivity. We propose a novel ASR framework, PI-Whisper, in this work and show how it can improve an ASR's recognition capabilities adaptively by identifying different speakers' characteristics in real-time, how such an adaption can be performed incrementally without repetitive retraining, and how it can improve the equity and fairness for diverse speaker groups. More impressively, our proposed PI-Whisper framework attains all of these nice properties while still achieving state-of-the-art accuracy with up to 13.7% reduction of the word error rate (WER) with linear scalability with respect to computing resources. △ Less

Submitted 21 June, 2024; originally announced June 2024.

Comments: 11 pages, 3 figures

Electrical switching of Ising-superconducting nonreciprocity for quantum neuronal transistor

Authors: Junlin Xiong, Jiao Xie, Bin Cheng, Yudi Dai, Xinyu Cui, Lizheng Wang, Zenglin Liu, Ji Zhou, Naizhou Wang, Xianghan Xu, Xianhui Chen, Sang-Wook Cheong, Shi-Jun Liang, Feng Miao

Abstract: Nonreciprocal quantum transport effect is mainly governed by the symmetry breaking of the material systems and is gaining extensive attention in condensed matter physics. Realizing electrical switching of the polarity of the nonreciprocal transport without external magnetic field is essential to the development of nonreciprocal quantum devices. However, electrical switching of superconducting nonr… ▽ More Nonreciprocal quantum transport effect is mainly governed by the symmetry breaking of the material systems and is gaining extensive attention in condensed matter physics. Realizing electrical switching of the polarity of the nonreciprocal transport without external magnetic field is essential to the development of nonreciprocal quantum devices. However, electrical switching of superconducting nonreciprocity remains yet to be achieved. Here, we report the observation of field-free electrical switching of nonreciprocal Ising superconductivity in Fe3GeTe2/NbSe2 van der Waals (vdW) heterostructure. By taking advantage of this electrically switchable superconducting nonreciprocity, we demonstrate a proof-of-concept nonreciprocal quantum neuronal transistor, which allows for implementing the XOR logic gate and faithfully emulating biological functionality of a cortical neuron in the brain. Our work provides a promising pathway to realize field-free and electrically switchable nonreciprocity of quantum transport and demonstrate its potential in exploring neuromorphic quantum devices with both functionality and performance beyond the traditional devices. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Journal ref: Nature Communications 15, 4953 (2024)

D2O: Dynamic Discriminative Operations for Efficient Generative Inference of Large Language Models

Authors: Zhongwei Wan, Xinjian Wu, Yu Zhang, Yi Xin, Chaofan Tao, Zhihong Zhu, Xin Wang, Siqi Luo, Jing Xiong, Mi Zhang

Abstract: Efficient inference in Large Language Models (LLMs) is impeded by the growing memory demands of key-value (KV) caching, especially for longer sequences. Traditional KV cache eviction strategies, which prioritize less critical KV-pairs based on attention scores, often degrade generation quality, leading to issues such as context loss or hallucinations. To address this, we introduce Dynamic Discrimi… ▽ More Efficient inference in Large Language Models (LLMs) is impeded by the growing memory demands of key-value (KV) caching, especially for longer sequences. Traditional KV cache eviction strategies, which prioritize less critical KV-pairs based on attention scores, often degrade generation quality, leading to issues such as context loss or hallucinations. To address this, we introduce Dynamic Discriminative Operations (D2O), a novel method that utilizes two-level discriminative strategies to optimize KV cache size without fine-tuning, while preserving essential context. Initially, by observing varying densities of attention weights between shallow and deep layers, we use this insight to determine which layers should avoid excessive eviction to minimize information loss. Subsequently, for the eviction strategy in each layer, D2O innovatively incorporates a compensation mechanism that maintains a similarity threshold to re-discriminate the importance of previously discarded tokens, determining whether they should be recalled and merged with similar tokens. Our approach not only achieves significant memory savings and enhances inference throughput by more than 3 times but also maintains high-quality long-text generation. Extensive experiments across various benchmarks and LLM architectures have demonstrated that D2O significantly enhances performance with a constrained KV cache budget. △ Less

Submitted 23 June, 2024; v1 submitted 18 June, 2024; originally announced June 2024.

Some quenched and annealed limit theorems of superprocesses in random environments

Authors: Zeteng Fan, Jieliang Hong, Jie Xiong

Abstract: Let $X=(X_t, t\geq 0)$ be a superprocess in a random environment described by a Gaussian noise $W=\{W(t,x), t\geq 0, x\in \mathbb{R}^d\}$ white in time and colored in space with correlation kernel $g(x,y)$. When $d\geq 3$, under the condition that the correlation function $g(x,y)$ is bounded above by some appropriate function $\bar{g}(x-y)$, we present the quenched and annealed Strong Law of Large… ▽ More Let $X=(X_t, t\geq 0)$ be a superprocess in a random environment described by a Gaussian noise $W=\{W(t,x), t\geq 0, x\in \mathbb{R}^d\}$ white in time and colored in space with correlation kernel $g(x,y)$. When $d\geq 3$, under the condition that the correlation function $g(x,y)$ is bounded above by some appropriate function $\bar{g}(x-y)$, we present the quenched and annealed Strong Law of Large Numbers and the Central Limit Theorems regarding the weighted occupation measure $\int_0^t X_s ds$ as $t\to \infty$. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: 36 pages

MSC Class: 60H15; 60G57; 60J80

Mean-field stochastic linear quadratic control problem with random coefficients

Authors: Jie Xiong, Wen Xu

Abstract: In this paper, we first prove that the mean-field stochastic linear quadratic (MFSLQ) control problem with random coefficients has a unique optimal control and derive a preliminary stochastic maximum principle to characterize this optimal control by an optimality system. However, because of the term of the form $\mathbb{E}[A(\cdot)X(\cdot)] $ in the adjoint equation, which cannot be represented in… ▽ More In this paper, we first prove that the mean-field stochastic linear quadratic (MFSLQ) control problem with random coefficients has a unique optimal control and derive a preliminary stochastic maximum principle to characterize this optimal control by an optimality system. However, because of the term of the form $\mathbb{E}[A(\cdot)X(\cdot)] $ in the adjoint equation, which cannot be represented in the form $\mathbb{E}[A(\cdot)]\mathbb{E} [X(\cdot)] $, we cannot solve this optimality system explicitly. To this end, we decompose the MFSLQ control problem into two constrained SLQ control problems without the mean-field terms. These constrained SLQ control problems can be solved explicitly by an extended LaGrange multiplier method developed in this article. △ Less

Submitted 30 July, 2024; v1 submitted 7 June, 2024; originally announced June 2024.

Empirical Guidelines for Deploying LLMs onto Resource-constrained Edge Devices

Authors: Ruiyang Qin, Dancheng Liu, Zheyu Yan, Zhaoxuan Tan, Zixuan Pan, Zhenge Jia, Meng Jiang, Ahmed Abbasi, Jinjun Xiong, Yiyu Shi

Abstract: The scaling laws have become the de facto guidelines for designing large language models (LLMs), but they were studied under the assumption of unlimited computing resources for both training and inference. As LLMs are increasingly used as personalized intelligent assistants, their customization (i.e., learning through fine-tuning) and deployment onto resource-constrained edge devices will become m… ▽ More The scaling laws have become the de facto guidelines for designing large language models (LLMs), but they were studied under the assumption of unlimited computing resources for both training and inference. As LLMs are increasingly used as personalized intelligent assistants, their customization (i.e., learning through fine-tuning) and deployment onto resource-constrained edge devices will become more and more prevalent. An urging but open question is how a resource-constrained computing environment would affect the design choices for a personalized LLM. We study this problem empirically in this work. In particular, we consider the tradeoffs among a number of key design factors and their intertwined impacts on learning efficiency and accuracy. The factors include the learning methods for LLM customization, the amount of personalized data used for learning customization, the types and sizes of LLMs, the compression methods of LLMs, the amount of time afforded to learn, and the difficulty levels of the target use cases. Through extensive experimentation and benchmarking, we draw a number of surprisingly insightful guidelines for deploying LLMs onto resource-constrained devices. For example, an optimal choice between parameter learning and RAG may vary depending on the difficulty of the downstream task, the longer fine-tuning time does not necessarily help the model, and a compressed LLM may be a better choice than an uncompressed LLM to learn from limited personalized data. △ Less

Submitted 13 June, 2024; v1 submitted 6 June, 2024; originally announced June 2024.

Comments: Benckmarking paper

AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathways

Authors: Zehang Deng, Yongjian Guo, Changzhou Han, Wanlun Ma, Junwu Xiong, Sheng Wen, Yang Xiang

Abstract: An Artificial Intelligence (AI) agent is a software entity that autonomously performs tasks or makes decisions based on pre-defined objectives and data inputs. AI agents, capable of perceiving user inputs, reasoning and planning tasks, and executing actions, have seen remarkable advancements in algorithm development and task performance. However, the security challenges they pose remain under-expl… ▽ More An Artificial Intelligence (AI) agent is a software entity that autonomously performs tasks or makes decisions based on pre-defined objectives and data inputs. AI agents, capable of perceiving user inputs, reasoning and planning tasks, and executing actions, have seen remarkable advancements in algorithm development and task performance. However, the security challenges they pose remain under-explored and unresolved. This survey delves into the emerging security threats faced by AI agents, categorizing them into four critical knowledge gaps: unpredictability of multi-step user inputs, complexity in internal executions, variability of operational environments, and interactions with untrusted external entities. By systematically reviewing these threats, this paper highlights both the progress made and the existing limitations in safeguarding AI agents. The insights provided aim to inspire further research into addressing the security threats associated with AI agents, thereby fostering the development of more robust and secure AI agent applications. △ Less

Submitted 3 June, 2024; originally announced June 2024.

Comments: ACM Computing Survey

Unlocking Guidance for Discrete State-Space Diffusion and Flow Models

Authors: Hunter Nisonoff, Junhao Xiong, Stephan Allenspach, Jennifer Listgarten

Abstract: Generative models on discrete state-spaces have a wide range of potential applications, particularly in the domain of natural sciences. In continuous state-spaces, controllable and flexible generation of samples with desired properties has been realized using guidance on diffusion and flow models. However, these guidance approaches are not readily amenable to discrete state-space models. Consequen… ▽ More Generative models on discrete state-spaces have a wide range of potential applications, particularly in the domain of natural sciences. In continuous state-spaces, controllable and flexible generation of samples with desired properties has been realized using guidance on diffusion and flow models. However, these guidance approaches are not readily amenable to discrete state-space models. Consequently, we introduce a general and principled method for applying guidance on such models. Our method depends on leveraging continuous-time Markov processes on discrete state-spaces, which unlocks computational tractability for sampling from a desired guided distribution. We demonstrate the utility of our approach, Discrete Guidance, on a range of applications including guided generation of images, small-molecules, DNA sequences and protein sequences. △ Less

Submitted 31 July, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

Important node identification for complex networks based on improved Electre Multi-Attribute fusion

Authors: Qi Cao, Yurong Song, Min Li, Ruqi Li, Hongbo Qu, Guo-Ping Jiang, Jinye Xiong

Abstract: Influence maximization problem involves selecting a subset of seed nodes within a social network to maximize information spread under a given diffusion model, so how to identify the important nodes is the problem to be considered in this paper. Due to the great differences in the reality of the network, a class of multi-attribute decision fusion methods is often used to solve this problem. Electre… ▽ More Influence maximization problem involves selecting a subset of seed nodes within a social network to maximize information spread under a given diffusion model, so how to identify the important nodes is the problem to be considered in this paper. Due to the great differences in the reality of the network, a class of multi-attribute decision fusion methods is often used to solve this problem. Electre is mostly used to solve the problems of investment order, benefit, and risk assessment of projects in economics, which supports the decision maker to make choices by comparing the differences between a set of alternatives. In this paper, we propose a multi-attribute decision fusion method named SK-E, which construct local and global metrics for different networks, use the improved Electre to make decision fusion between local and global metrics of nodes, to get the optimal weight between local and global metrics, and then identify the important nodes. The proposed method demonstrates superior accuracy compared to other methods, as evaluated through three experiments: the SIR epidemic model, the independent cascade model, and constraint efficiency. These experiments were conducted across six different real networks selected as the experimental dataset. △ Less

Submitted 3 June, 2024; originally announced June 2024.

The First Photometric Analysis of Two Low Mass Ratio Contact Binary Systems In TESS Survey

Authors: Qiyuan Cheng, Jianping XIong, Xu Ding, Kaifan Ji, Jiao Li, Chao Liu, Jiangdan Li, Jingxiao Luo, Xin Lyu, Zhanwen Han, Xuefei Chen

Abstract: Low mass-ratio (q) contact binary systems are progenitors of stellar mergers such as blue straggles (BS) or fast-rotating FK Com stars. In this study, we present the first light curve analysis of two newly identified low mass-ratio contact binary systems, TIC 55007847 and TIC 63597006, that are identified from TESS. Both stars are classified as A-subtype contact binaries. We obtained the precise o… ▽ More Low mass-ratio (q) contact binary systems are progenitors of stellar mergers such as blue straggles (BS) or fast-rotating FK Com stars. In this study, we present the first light curve analysis of two newly identified low mass-ratio contact binary systems, TIC 55007847 and TIC 63597006, that are identified from TESS. Both stars are classified as A-subtype contact binaries. We obtained the precise orbit periods for the two objects by using the O-C method, i.e. P=0.6117108 d for TIC 55007847 and P=0.7008995 d for TIC 63597006, respectively, and found an obvious periodic signal in the O-C curve of TIC 63597006. We suggest that the periodic signal comes from a third body. We further use the Markov Chain Monte Carlo (MCMC) method with PHOEBE to derive the photometric solutions for the two binaries. The photometric solution for this object shows that the contribution of the third body is about 6%. Our analysis revealed that TIC 55007847 has an extremely low mass ratio of q=0.08. By calculating the ratio of spin angular momentum to the orbital angular momentum Js/Jo, we found that TIC 55007847 is very close to the instability threshold with Js/Jo = 0.31, indicating that it may merge into a single, fast-rotating star in the future. For TIC 63597006, q=0.14 and Js/Jo=0.15. This object is in a relatively stable evolutionary status at present. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Vision Language Models for Spreadsheet Understanding: Challenges and Opportunities

Authors: Shiyu Xia, Junyu Xiong, Haoyu Dong, Jianbo Zhao, Yuzhang Tian, Mengyu Zhou, Yeye He, Shi Han, Dongmei Zhang

Abstract: This paper explores capabilities of Vision Language Models on spreadsheet comprehension. We propose three self-supervised challenges with corresponding evaluation metrics to comprehensively evaluate VLMs on Optical Character Recognition (OCR), spatial perception, and visual format recognition. Additionally, we utilize the spreadsheet table detection task to assess the overall performance of VLMs b… ▽ More This paper explores capabilities of Vision Language Models on spreadsheet comprehension. We propose three self-supervised challenges with corresponding evaluation metrics to comprehensively evaluate VLMs on Optical Character Recognition (OCR), spatial perception, and visual format recognition. Additionally, we utilize the spreadsheet table detection task to assess the overall performance of VLMs by integrating these challenges. To probe VLMs more finely, we propose three spreadsheet-to-image settings: column width adjustment, style change, and address augmentation. We propose variants of prompts to address the above tasks in different settings. Notably, to leverage the strengths of VLMs in understanding text rather than two-dimensional positioning, we propose to decode cell values on the four boundaries of the table in spreadsheet boundary detection. Our findings reveal that VLMs demonstrate promising OCR capabilities but produce unsatisfactory results due to cell omission and misalignment, and they notably exhibit insufficient spatial and format recognition skills, motivating future work to enhance VLMs' spreadsheet data comprehension capabilities using our methods to generate extensive spreadsheet-image pairs in various settings. △ Less

Submitted 8 August, 2024; v1 submitted 25 May, 2024; originally announced May 2024.

CAPE: Context-Adaptive Positional Encoding for Length Extrapolation

Authors: Chuanyang Zheng, Yihang Gao, Han Shi, Minbin Huang, Jingyao Li, Jing Xiong, Xiaozhe Ren, Michael Ng, Xin Jiang, Zhenguo Li, Yu Li

Abstract: Positional encoding plays a crucial role in transformers, significantly impacting model performance and length generalization. Prior research has introduced absolute positional encoding (APE) and relative positional encoding (RPE) to distinguish token positions in given sequences. However, both APE and RPE remain fixed after model training regardless of input data, limiting their adaptability and… ▽ More Positional encoding plays a crucial role in transformers, significantly impacting model performance and length generalization. Prior research has introduced absolute positional encoding (APE) and relative positional encoding (RPE) to distinguish token positions in given sequences. However, both APE and RPE remain fixed after model training regardless of input data, limiting their adaptability and flexibility. Hence, we expect that the desired positional encoding should be context-adaptive and can be dynamically adjusted with the given attention. In this paper, we propose a Context-Adaptive Positional Encoding (CAPE) method, which dynamically and semantically adjusts based on input context and learned fixed priors. Experimental validation on real-world datasets (Arxiv, Books3, and CHE) demonstrates that CAPE enhances model performances in terms of trained length and length generalization, where the improvements are statistically significant. The model visualization suggests that our model can keep both local and anti-local information. Finally, we successfully train the model on sequence length 128 and achieve better performance at evaluation sequence length 8192, compared with other static positional encoding methods, revealing the benefit of the adaptive positional encoding method. △ Less

Submitted 23 May, 2024; originally announced May 2024.

Comments: Technical Report

Infinite-Dimensional Feature Interaction

Authors: Chenhui Xu, Fuxun Yu, Maoliang Li, Zihao Zheng, Zirui Xu, Jinjun Xiong, Xiang Chen

Abstract: The past neural network design has largely focused on feature representation space dimension and its capacity scaling (e.g., width, depth), but overlooked the feature interaction space scaling. Recent advancements have shown shifted focus towards element-wise multiplication to facilitate higher-dimensional feature interaction space for better information transformation. Despite this progress, mu… ▽ More The past neural network design has largely focused on feature representation space dimension and its capacity scaling (e.g., width, depth), but overlooked the feature interaction space scaling. Recent advancements have shown shifted focus towards element-wise multiplication to facilitate higher-dimensional feature interaction space for better information transformation. Despite this progress, multiplications predominantly capture low-order interactions, thus remaining confined to a finite-dimensional interaction space. To transcend this limitation, classic kernel methods emerge as a promising solution to engage features in an infinite-dimensional space. We introduce InfiNet, a model architecture that enables feature interaction within an infinite-dimensional space created by RBF kernel. Our experiments reveal that InfiNet achieves new state-of-the-art, owing to its capability to leverage infinite-dimensional interactions, significantly enhancing model performance. △ Less

Submitted 9 June, 2024; v1 submitted 22 May, 2024; originally announced May 2024.

Hummer: Towards Limited Competitive Preference Dataset

Authors: Li Jiang, Yusen Wu, Junwu Xiong, Jingqing Ruan, Yichuan Ding, Qingpei Guo, Zujie Wen, Jun Zhou, Xiaotie Deng

Abstract: Preference datasets are essential for incorporating human preferences into pre-trained language models, playing a key role in the success of Reinforcement Learning from Human Feedback. However, these datasets often demonstrate conflicting alignment objectives, leading to increased vulnerability to jailbreak attacks and challenges in adapting downstream tasks to prioritize specific alignment object… ▽ More Preference datasets are essential for incorporating human preferences into pre-trained language models, playing a key role in the success of Reinforcement Learning from Human Feedback. However, these datasets often demonstrate conflicting alignment objectives, leading to increased vulnerability to jailbreak attacks and challenges in adapting downstream tasks to prioritize specific alignment objectives without negatively impacting others. In this work, we introduce a novel statistical metric, Alignment Dimension Conflict, to quantify the degree of conflict within preference datasets. We then present \texttt{Hummer} and its fine-grained variant, \texttt{Hummer-F}, as innovative pairwise preference datasets with reduced-conflict alignment objectives. \texttt{Hummer} is built based on UltraFeedback and is enhanced by AI feedback from GPT-4, marking as the first preference dataset aimed at reducing the competition between alignment objectives. Furthermore, we develop reward models, HummerRM and HummerRM-F, which employ a hybrid sampling approach to balance diverse alignment objectives effectively. This sampling method positions HummerRM as an ideal model for domain-specific further fine-tuning and reducing vulnerabilities to attacks. △ Less

Submitted 6 August, 2024; v1 submitted 19 May, 2024; originally announced May 2024.

Journal ref: COLM 2024

RSHazeDiff: A Unified Fourier-aware Diffusion Model for Remote Sensing Image Dehazing

Authors: Jiamei Xiong, Xuefeng Yan, Yongzhen Wang, Wei Zhao, Xiao-Ping Zhang, Mingqiang Wei

Abstract: Haze severely degrades the visual quality of remote sensing images and hampers the performance of automotive navigation, intelligent monitoring, and urban management. The emerging denoising diffusion probabilistic model (DDPM) exhibits the significant potential for dense haze removal with its strong generation ability. Since remote sensing images contain extensive small-scale texture structures, i… ▽ More Haze severely degrades the visual quality of remote sensing images and hampers the performance of automotive navigation, intelligent monitoring, and urban management. The emerging denoising diffusion probabilistic model (DDPM) exhibits the significant potential for dense haze removal with its strong generation ability. Since remote sensing images contain extensive small-scale texture structures, it is important to effectively restore image details from hazy images. However, current wisdom of DDPM fails to preserve image details and color fidelity well, limiting its dehazing capacity for remote sensing images. In this paper, we propose a novel unified Fourier-aware diffusion model for remote sensing image dehazing, termed RSHazeDiff. From a new perspective, RSHazeDiff explores the conditional DDPM to improve image quality in dense hazy scenarios, and it makes three key contributions. First, RSHazeDiff refines the training phase of diffusion process by performing noise estimation and reconstruction constraints in a coarse-to-fine fashion. Thus, it remedies the unpleasing results caused by the simple noise estimation constraint in DDPM. Second, by taking the frequency information as important prior knowledge during iterative sampling steps, RSHazeDiff can preserve more texture details and color fidelity in dehazed images. Third, we design a global compensated learning module to utilize the Fourier transform to capture the global dependency features of input images, which can effectively mitigate the effects of boundary artifacts when processing fixed-size patches. Experiments on both synthetic and real-world benchmarks validate the favorable performance of RSHazeDiff over multiple state-of-the-art methods. Source code will be released at https://github.com/jm-xiong/RSHazeDiff. △ Less

Submitted 15 May, 2024; originally announced May 2024.

Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding

Authors: Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, Dayou Chen, Jiajun He, Jiahao Li, Wenyue Li, Chen Zhang, Rongwei Quan, Jianxiang Lu, Jiabin Huang, Xiaoyan Yuan, Xiaoxiao Zheng, Yixuan Li, Jihong Zhang, Chao Zhang, Meng Chen, Jie Liu , et al. (20 additional authors not shown)

Abstract: We present Hunyuan-DiT, a text-to-image diffusion transformer with fine-grained understanding of both English and Chinese. To construct Hunyuan-DiT, we carefully design the transformer structure, text encoder, and positional encoding. We also build from scratch a whole data pipeline to update and evaluate data for iterative model optimization. For fine-grained language understanding, we train a Mu… ▽ More We present Hunyuan-DiT, a text-to-image diffusion transformer with fine-grained understanding of both English and Chinese. To construct Hunyuan-DiT, we carefully design the transformer structure, text encoder, and positional encoding. We also build from scratch a whole data pipeline to update and evaluate data for iterative model optimization. For fine-grained language understanding, we train a Multimodal Large Language Model to refine the captions of the images. Finally, Hunyuan-DiT can perform multi-turn multimodal dialogue with users, generating and refining images according to the context. Through our holistic human evaluation protocol with more than 50 professional human evaluators, Hunyuan-DiT sets a new state-of-the-art in Chinese-to-image generation compared with other open-source models. Code and pretrained models are publicly available at github.com/Tencent/HunyuanDiT △ Less

Submitted 14 May, 2024; originally announced May 2024.

Comments: Project Page: https://dit.hunyuan.tencent.com/

A Timely Survey on Vision Transformer for Deepfake Detection

Authors: Zhikan Wang, Zhongyao Cheng, Jiajie Xiong, Xun Xu, Tianrui Li, Bharadwaj Veeravalli, Xulei Yang

Abstract: In recent years, the rapid advancement of deepfake technology has revolutionized content creation, lowering forgery costs while elevating quality. However, this progress brings forth pressing concerns such as infringements on individual rights, national security threats, and risks to public safety. To counter these challenges, various detection methodologies have emerged, with Vision Transformer (… ▽ More In recent years, the rapid advancement of deepfake technology has revolutionized content creation, lowering forgery costs while elevating quality. However, this progress brings forth pressing concerns such as infringements on individual rights, national security threats, and risks to public safety. To counter these challenges, various detection methodologies have emerged, with Vision Transformer (ViT)-based approaches showcasing superior performance in generality and efficiency. This survey presents a timely overview of ViT-based deepfake detection models, categorized into standalone, sequential, and parallel architectures. Furthermore, it succinctly delineates the structure and characteristics of each model. By analyzing existing research and addressing future directions, this survey aims to equip researchers with a nuanced understanding of ViT's pivotal role in deepfake detection, serving as a valuable reference for both academic and practical pursuits in this domain. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Robust Implementation of Retrieval-Augmented Generation on Edge-based Computing-in-Memory Architectures

Authors: Ruiyang Qin, Zheyu Yan, Dewen Zeng, Zhenge Jia, Dancheng Liu, Jianbo Liu, Zhi Zheng, Ningyuan Cao, Kai Ni, Jinjun Xiong, Yiyu Shi

Abstract: Large Language Models (LLMs) deployed on edge devices learn through fine-tuning and updating a certain portion of their parameters. Although such learning methods can be optimized to reduce resource utilization, the overall required resources remain a heavy burden on edge devices. Instead, Retrieval-Augmented Generation (RAG), a resource-efficient LLM learning method, can improve the quality of th… ▽ More Large Language Models (LLMs) deployed on edge devices learn through fine-tuning and updating a certain portion of their parameters. Although such learning methods can be optimized to reduce resource utilization, the overall required resources remain a heavy burden on edge devices. Instead, Retrieval-Augmented Generation (RAG), a resource-efficient LLM learning method, can improve the quality of the LLM-generated content without updating model parameters. However, the RAG-based LLM may involve repetitive searches on the profile data in every user-LLM interaction. This search can lead to significant latency along with the accumulation of user data. Conventional efforts to decrease latency result in restricting the size of saved user data, thus reducing the scalability of RAG as user data continuously grows. It remains an open question: how to free RAG from the constraints of latency and scalability on edge devices? In this paper, we propose a novel framework to accelerate RAG via Computing-in-Memory (CiM) architectures. It accelerates matrix multiplications by performing in-situ computation inside the memory while avoiding the expensive data transfer between the computing unit and memory. Our framework, Robust CiM-backed RAG (RoCR), utilizing a novel contrastive learning-based training method and noise-aware training, can enable RAG to efficiently search profile data with CiM. To the best of our knowledge, this is the first work utilizing CiM to accelerate RAG. △ Less

Submitted 7 May, 2024; originally announced May 2024.

QuadraNet V2: Efficient and Sustainable Training of High-Order Neural Networks with Quadratic Adaptation

Authors: Chenhui Xu, Xinyao Wang, Fuxun Yu, Jinjun Xiong, Xiang Chen

Abstract: Machine learning is evolving towards high-order models that necessitate pre-training on extensive datasets, a process associated with significant overheads. Traditional models, despite having pre-trained weights, are becoming obsolete due to architectural differences that obstruct the effective transfer and initialization of these weights. To address these challenges, we introduce a novel framewor… ▽ More Machine learning is evolving towards high-order models that necessitate pre-training on extensive datasets, a process associated with significant overheads. Traditional models, despite having pre-trained weights, are becoming obsolete due to architectural differences that obstruct the effective transfer and initialization of these weights. To address these challenges, we introduce a novel framework, QuadraNet V2, which leverages quadratic neural networks to create efficient and sustainable high-order learning models. Our method initializes the primary term of the quadratic neuron using a standard neural network, while the quadratic term is employed to adaptively enhance the learning of data non-linearity or shifts. This integration of pre-trained primary terms with quadratic terms, which possess advanced modeling capabilities, significantly augments the information characterization capacity of the high-order network. By utilizing existing pre-trained weights, QuadraNet V2 reduces the required GPU hours for training by 90\% to 98.4\% compared to training from scratch, demonstrating both efficiency and effectiveness. △ Less

Submitted 8 May, 2024; v1 submitted 6 May, 2024; originally announced May 2024.

From Keyboard to Chatbot: An AI-powered Integration Platform with Large-Language Models for Teaching Computational Thinking for Young Children

Authors: Changjae Lee, Jinjun Xiong

Abstract: Teaching programming in early childhood (4-9) to enhance computational thinking has gained popularity in the recent movement of computer science for all. However, current practices ignore some fundamental issues resulting from young children's developmental readiness, such as the sustained capability to keyboarding, the decomposition of complex tasks to small tasks, the need for intuitive mapping… ▽ More Teaching programming in early childhood (4-9) to enhance computational thinking has gained popularity in the recent movement of computer science for all. However, current practices ignore some fundamental issues resulting from young children's developmental readiness, such as the sustained capability to keyboarding, the decomposition of complex tasks to small tasks, the need for intuitive mapping from abstract programming to tangible outcomes, and the limited amount of screen time exposure. To address these issues in this paper, we present a novel methodology with an AI-powered integration platform to effectively teach computational thinking for young children. The system features a hybrid pedagogy that supports both the top-down and bottom-up approach for teaching computational thinking. Young children can describe their desired task in natural language, while the system can respond with an easy-to-understand program consisting of the right level of decomposed sub-tasks. A tangible robot can immediately execute the decomposed program and demonstrate the program's outcomes to young children. The system is equipped with an intelligent chatbot that can interact with young children through natural languages, and children can speak to the chatbot to complete all the needed programming tasks, while the chatbot orchestrates the execution of the program onto the robot. This would completely eliminates the need of keyboards for young children to program. By developing such a system, we aim to make the concept of computational thinking more accessible to young children, fostering a natural understanding of programming concepts without the need of explicit programming skills. Through the interactive experience provided by the robotic agent, our system seeks to engage children in an effective manner, contributing to the field of educational technology for early childhood computer science education. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: 26 pages, 11 figures

Social Optima of Linear Forward-Backward Stochastic System

Authors: Guangchen Wang, Shujun Wang, Jie Xiong

Abstract: A linear quadratic (LQ) stochastic optimization system involving large population, which is driven by forward-backward stochastic differential equation (FBSDE), is investigated in this paper. Agents cooperate with each other to minimize the so-called social objective, which is rather different from mean field (MF) game. Employing forward-backward person-by-person optimality principle, we derive an… ▽ More A linear quadratic (LQ) stochastic optimization system involving large population, which is driven by forward-backward stochastic differential equation (FBSDE), is investigated in this paper. Agents cooperate with each other to minimize the so-called social objective, which is rather different from mean field (MF) game. Employing forward-backward person-by-person optimality principle, we derive an auxiliary LQ control problem by decentralized information. A decentralized strategy is obtained by virtue of an MF-type forward-backward stochastic differential equation consistency condition. Applying Riccati equation decoupling method, we solve the consistency condition system. We also verify the asymptotic social optimality in this framework. △ Less

Submitted 29 April, 2024; originally announced April 2024.

Comments: 30 pages

Dose rate dependence of TID damage to 65 nm CMOS transistors in X-ray irradiations of the ATLAS ITk Pixel ASIC (ITkPix)

Authors: Daniela Bortoletto, Aleksandra Dimitrievska, Maurice Garcia-Sciveres, Timon Heim, Maria Mironova, Richard Plackett, Ian Shipsey, Junwen Xiong

Abstract: The ATLAS Inner Tracker (ITk) upgrade for the High-Luminosity LHC (HL-LHC) requires a radiation-tolerant pixel readout chip, which must withstand a total ionising dose (TID) of up to 1 Grad. The readout ASIC for the ITk upgrade has been designed by the RD53 collaboration using 65 nm CMOS technology. In order to characterise the radiation tolerance of the chip digital logic, the RD53 ASICs include… ▽ More The ATLAS Inner Tracker (ITk) upgrade for the High-Luminosity LHC (HL-LHC) requires a radiation-tolerant pixel readout chip, which must withstand a total ionising dose (TID) of up to 1 Grad. The readout ASIC for the ITk upgrade has been designed by the RD53 collaboration using 65 nm CMOS technology. In order to characterise the radiation tolerance of the chip digital logic, the RD53 ASICs include ring oscillators, which can be used to measure gate delay degradation. Extensive X-ray irradiation studies of the ring oscillators have been performed on the ITk Pixel pre-production readout ASIC, ITkPixV1. A dependence of radiation damage on dose rate has been observed in 65 nm CMOS technology. This paper aims to quantify the dose rate dependence of TID damage to the ITkPix ring oscillators and, therefore, the ITkPix ASIC digital logic. X-ray irradiations at different dose rates between 20 krad/h and 30 Mrad/h are compared. A dose rate dependence is observed, with 2-3 times more damage at the lowest dose rate of 20 krad/h, compared to 4 Mrad/h. The dose rate dependence was also observed to be dependent on transistor size and type. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Two system transformation data-driven algorithms for linear quadratic mean-field games

Authors: Xun Li, Guangchen Wang, Yu Wang, Jie Xiong, Heng Zhang

Abstract: This paper studies a class of continuous-time linear quadratic (LQ) mean-field game problems. We develop two system transformation data-driven algorithms to approximate the decentralized strategies of the LQ mean-field games. The main feature of the obtained data-driven algorithms is that they eliminate the requirement on all system matrices. First, we transform the original stochastic system into… ▽ More This paper studies a class of continuous-time linear quadratic (LQ) mean-field game problems. We develop two system transformation data-driven algorithms to approximate the decentralized strategies of the LQ mean-field games. The main feature of the obtained data-driven algorithms is that they eliminate the requirement on all system matrices. First, we transform the original stochastic system into an ordinary differential equation (ODE). Subsequently, we construct some Kronecker product-based matrices by the input/state data of the ODE. By virtue of these matrices, we implement a model-based policy iteration (PI) algorithm and a model-based value iteration (VI) algorithm in a data-driven fashion. In addition, we also demonstrate the convergence of these two data-driven algorithms under some mild conditions. Finally, we illustrate the practicality of our algorithms via two numerical examples. △ Less

Submitted 16 April, 2024; originally announced April 2024.

A born ultramassive white dwarf-hot subdwarf super-Chandrasekhar candidate

Authors: Changqing Luo, Jiao Li, Chuanjie Zheng, Dongdong Liu, Zhenwei Li, Yangping Luo, Peter Nemeth, Bo Zhang, Jianping Xiong, Bo Wang, Song Wang, Yu Bai, Qingzheng Li, Pei Wang, Zhanwen Han, Jifeng Liu, Yang Huang, Xuefei Chen, Chao Liu

Abstract: Although supernovae is a well-known endpoint of an accreting white dwarf, alternative theoretical possibilities has been discussing broadly, such as the accretion-induced collapse (AIC) event as the endpoint of oxygen-neon (ONe) white dwarfs, either accreting up to or merging to excess the Chandrasekhar limit (the maximum mass of a stable white dwarf). AIC is an important channel to form neutron s… ▽ More Although supernovae is a well-known endpoint of an accreting white dwarf, alternative theoretical possibilities has been discussing broadly, such as the accretion-induced collapse (AIC) event as the endpoint of oxygen-neon (ONe) white dwarfs, either accreting up to or merging to excess the Chandrasekhar limit (the maximum mass of a stable white dwarf). AIC is an important channel to form neutron stars, especially for those unusual systems, which are hardly produced by core-collapse supernovae. However, the observational evidences for this theoretical predicted event and its progenitor are all very limited. In all of the known progenitors, white dwarfs increase in mass by accretion. Here, we report the discovery of an intriguing binary system Lan 11, consisted of a stripped core-helium-burning hot subdwarf and an unseen compact object of 1.08 to 1.35 $M_{\odot}$. Our binary population synthesis calculations, along with the absence of detection from the deep radio observations of the Five-hundred-meter Aperture Spherical Radio Telescope, strongly suggest that the latter is an ONe white dwarf. The total mass of this binary is 1.67 to 1.92 $M_{\odot}$}, significantly excessing the Chandrasekhar limit. The reproduction of its evolutionary history indicates that the unique system has undergone two phases of common envelope ejections, implying a born nature of this massive ONe white dwarf rather than an accretion growth from its companion. These results, together with short orbital period of this binary (3.65 hours), suggest that this system will merge in 500-540 Myr, largely triggering an AIC event, although the possibility of type Ia supernova cannot be fully ruled out. This finding greatly provides valuable constraints on our understanding of stellar endpoints, whatever leading to an AIC or a supernova. △ Less

Submitted 7 April, 2024; originally announced April 2024.

Comments: 25 pages, 14 figures

Multi-Level Label Correction by Distilling Proximate Patterns for Semi-supervised Semantic Segmentation

Authors: Hui Xiao, Yuting Hong, Li Dong, Diqun Yan, Jiayan Zhuang, Junjie Xiong, Dongtai Liang, Chengbin Peng

Abstract: Semi-supervised semantic segmentation relieves the reliance on large-scale labeled data by leveraging unlabeled data. Recent semi-supervised semantic segmentation approaches mainly resort to pseudo-labeling methods to exploit unlabeled data. However, unreliable pseudo-labeling can undermine the semi-supervision processes. In this paper, we propose an algorithm called Multi-Level Label Correction (… ▽ More Semi-supervised semantic segmentation relieves the reliance on large-scale labeled data by leveraging unlabeled data. Recent semi-supervised semantic segmentation approaches mainly resort to pseudo-labeling methods to exploit unlabeled data. However, unreliable pseudo-labeling can undermine the semi-supervision processes. In this paper, we propose an algorithm called Multi-Level Label Correction (MLLC), which aims to use graph neural networks to capture structural relationships in Semantic-Level Graphs (SLGs) and Class-Level Graphs (CLGs) to rectify erroneous pseudo-labels. Specifically, SLGs represent semantic affinities between pairs of pixel features, and CLGs describe classification consistencies between pairs of pixel labels. With the support of proximate pattern information from graphs, MLLC can rectify incorrectly predicted pseudo-labels and can facilitate discriminative feature representations. We design an end-to-end network to train and perform this effective label corrections mechanism. Experiments demonstrate that MLLC can significantly improve supervised baselines and outperforms state-of-the-art approaches in different scenarios on Cityscapes and PASCAL VOC 2012 datasets. Specifically, MLLC improves the supervised baseline by at least 5% and 2% with DeepLabV2 and DeepLabV3+ respectively under different partition protocols. △ Less

Submitted 9 April, 2024; v1 submitted 2 April, 2024; originally announced April 2024.

Comments: 12 pages, 8 figures. IEEE Transactions on Multimedia, 2024

CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians

Authors: Avinash Paliwal, Wei Ye, Jinhui Xiong, Dmytro Kotovenko, Rakesh Ranjan, Vikas Chandra, Nima Khademi Kalantari

Abstract: The field of 3D reconstruction from images has rapidly evolved in the past few years, first with the introduction of Neural Radiance Field (NeRF) and more recently with 3D Gaussian Splatting (3DGS). The latter provides a significant edge over NeRF in terms of the training and inference speed, as well as the reconstruction quality. Although 3DGS works well for dense input images, the unstructured p… ▽ More The field of 3D reconstruction from images has rapidly evolved in the past few years, first with the introduction of Neural Radiance Field (NeRF) and more recently with 3D Gaussian Splatting (3DGS). The latter provides a significant edge over NeRF in terms of the training and inference speed, as well as the reconstruction quality. Although 3DGS works well for dense input images, the unstructured point-cloud like representation quickly overfits to the more challenging setup of extremely sparse input images (e.g., 3 images), creating a representation that appears as a jumble of needles from novel views. To address this issue, we propose regularized optimization and depth-based initialization. Our key idea is to introduce a structured Gaussian representation that can be controlled in 2D image space. We then constraint the Gaussians, in particular their position, and prevent them from moving independently during optimization. Specifically, we introduce single and multiview constraints through an implicit convolutional decoder and a total variation loss, respectively. With the coherency introduced to the Gaussians, we further constrain the optimization through a flow-based loss function. To support our regularized optimization, we propose an approach to initialize the Gaussians using monocular depth estimates at each input view. We demonstrate significant improvements compared to the state-of-the-art sparse-view NeRF-based approaches on a variety of scenes. △ Less

Submitted 28 March, 2024; originally announced March 2024.

Comments: Project page: https://people.engr.tamu.edu/nimak/Papers/CoherentGS

Interfacial magnetic spin Hall effect in van der Waals Fe3GeTe2/MoTe2 heterostructure

Authors: Yudi Dai, Junlin Xiong, Yanfeng Ge, Bin Cheng, Lizheng Wang, Pengfei Wang, Zenglin Liu, Shengnan Yan, Cuiwei Zhang, Xianghan Xu, Youguo Shi, Sang-Wook Cheong, Cong Xiao, Shengyuan A. Yang, Shi-Jun Liang, Feng Miao

Abstract: The spin Hall effect (SHE) allows efficient generation of spin polarization or spin current through charge current and plays a crucial role in the development of spintronics. While SHE typically occurs in non-magnetic materials and is time-reversal even, exploring time-reversal-odd (T-odd) SHE, which couples SHE to magnetization in ferromagnetic materials, offers a new charge-spin conversion mecha… ▽ More The spin Hall effect (SHE) allows efficient generation of spin polarization or spin current through charge current and plays a crucial role in the development of spintronics. While SHE typically occurs in non-magnetic materials and is time-reversal even, exploring time-reversal-odd (T-odd) SHE, which couples SHE to magnetization in ferromagnetic materials, offers a new charge-spin conversion mechanism with new functionalities. Here, we report the observation of giant T-odd SHE in Fe3GeTe2/MoTe2 van der Waals heterostructure, representing a previously unidentified interfacial magnetic spin Hall effect (interfacial-MSHE). Through rigorous symmetry analysis and theoretical calculations, we attribute the interfacial-MSHE to a symmetry-breaking induced spin current dipole at the vdW interface. Furthermore, we show that this linear effect can be used for implementing multiply-accumulate operations and binary convolutional neural networks with cascaded multi-terminal devices. Our findings uncover an interfacial T-odd charge-spin conversion mechanism with promising potential for energy-efficient in-memory computing. △ Less

Submitted 26 March, 2024; originally announced March 2024.

Journal ref: Nature Communications 15, 1129 (2024)

Image Captioning in news report scenario

Authors: Tianrui Liu, Qi Cai, Changxin Xu, Bo Hong, Jize Xiong, Yuxin Qiao, Tsungwei Yang

Abstract: Image captioning strives to generate pertinent captions for specified images, situating itself at the crossroads of Computer Vision (CV) and Natural Language Processing (NLP). This endeavor is of paramount importance with far-reaching applications in recommendation systems, news outlets, social media, and beyond. Particularly within the realm of news reporting, captions are expected to encompass d… ▽ More Image captioning strives to generate pertinent captions for specified images, situating itself at the crossroads of Computer Vision (CV) and Natural Language Processing (NLP). This endeavor is of paramount importance with far-reaching applications in recommendation systems, news outlets, social media, and beyond. Particularly within the realm of news reporting, captions are expected to encompass detailed information, such as the identities of celebrities captured in the images. However, much of the existing body of work primarily centers around understanding scenes and actions. In this paper, we explore the realm of image captioning specifically tailored for celebrity photographs, illustrating its broad potential for enhancing news industry practices. This exploration aims to augment automated news content generation, thereby facilitating a more nuanced dissemination of information. Our endeavor shows a broader horizon, enriching the narrative in news reporting through a more intuitive image captioning framework. △ Less

Submitted 1 April, 2024; v1 submitted 24 March, 2024; originally announced March 2024.

Comments: 10 pages, 4 figures

Stochastic maximum principle for weighted mean-field system with jump

Authors: Yanyan Tang, Jie Xiong

Abstract: In this article, we consider a weighted mean-field control problem with jump-diffusion as its state process. The main difficulty is from the non-Lipschitz property of the coefficients. We overcome this difficulty by an $L_{p,q}$-estimate of the solution processes with a suitably chosen $p$ and $q$. Convex pertubation method combining with the aforementioned $L_{p,q}$-estimation method is utilized… ▽ More In this article, we consider a weighted mean-field control problem with jump-diffusion as its state process. The main difficulty is from the non-Lipschitz property of the coefficients. We overcome this difficulty by an $L_{p,q}$-estimate of the solution processes with a suitably chosen $p$ and $q$. Convex pertubation method combining with the aforementioned $L_{p,q}$-estimation method is utilized to derive the stochastic maximum principle for this control problem. A sufficient condition for the optimality is also given. △ Less

Submitted 24 March, 2024; originally announced March 2024.