Showing 1–50 of 345 results for author: Ju, L

Signatures of Chiral Superconductivity in Rhombohedral Graphene

Authors: Tonghang Han, Zhengguang Lu, Yuxuan Yao, Lihan Shi, Jixiang Yang, Junseok Seo, Shenyong Ye, Zhenghan Wu, Muyang Zhou, Haoyang Liu, Gang Shi, Zhenqi Hua, Kenji Watanabe, Takashi Taniguchi, Peng Xiong, Liang Fu, Long Ju

Abstract: Chiral superconductors are unconventional superconducting states that break time reversal symmetry spontaneously and typically feature Cooper pairing at non-zero angular momentum. Such states may host Majorana fermions and provide an important platform for topological physics research and fault-tolerant quantum computing. Despite of intensive search and prolonged studies of several candidate syste… ▽ More Chiral superconductors are unconventional superconducting states that break time reversal symmetry spontaneously and typically feature Cooper pairing at non-zero angular momentum. Such states may host Majorana fermions and provide an important platform for topological physics research and fault-tolerant quantum computing. Despite of intensive search and prolonged studies of several candidate systems, chiral superconductivity has remained elusive so far. Here we report the discovery of unconventional superconductivity in rhombohedral tetra-layer graphene. We observed two superconducting states in the gate-induced flat conduction bands with Tc up to 300 mK and charge density ne as low as 2.4*1011 cm-2, appearing robustly in three different devices, where electrons reside close to a proximate WSe2 layer, far away from WSe2, and in the absence of WSe2 respectively. Spontaneous time-reversal-symmetry-breaking (TRSB) due to electron's orbital motion is found, and several observations indicate the chiral nature of these superconducting states, including 1. In the superconducting state, Rxx shows fluctuations at zero magnetic field and magnetic hysteresis versus an out-of-plane magnetic field B, which are absent from all other superconductors; 2. one superconducting state develops within a spin- and valley-polarized quarter-metal phase, and is robust against the neighboring spin-valley-polarized quarter-metal state under B; 3. the normal states show anomalous Hall signals at zero magnetic field and magnetic hysteresis. We also observed a critical B > 0.9 Tesla, higher than any graphene superconductivity reported so far and indicates a strong-coupling superconductivity close the BCS-BEC crossover. Our observations establish a pure carbon material for the study of topological superconductivity, and pave the way to explore Majorana modes and topological quantum computing. △ Less

Submitted 27 August, 2024; originally announced August 2024.

Displacement field-controlled fractional Chern insulators and charge density waves in a graphene/hBN moiré superlattice

Authors: Samuel H. Aronson, Tonghang Han, Zhengguang Lu, Yuxuan Yao, Kenji Watanabe, Takashi Taniguchi, Long Ju, Raymond C. Ashoori

Abstract: Rhombohedral multilayer graphene, with its flat electronic bands and concentrated Berry curvature, is a promising material for the realization of correlated topological phases of matter. When aligned to an adjacent hexagonal boron nitride (hBN) layer, the graphene develops narrow minibands with non-trivial topology. By tuning an externally-applied electric displacement field, the conduction electr… ▽ More Rhombohedral multilayer graphene, with its flat electronic bands and concentrated Berry curvature, is a promising material for the realization of correlated topological phases of matter. When aligned to an adjacent hexagonal boron nitride (hBN) layer, the graphene develops narrow minibands with non-trivial topology. By tuning an externally-applied electric displacement field, the conduction electrons can either be pushed towards or away from the moiré superlattice. Motivated by the recent observation of the fractional quantum anomalous Hall effect (FQAHE) in the moiré-distant case, we study the opposite moiré-proximal case, where the superlattice potential is considerably stronger. We explore the physics within the moiré conduction bands through capacitance measurements that allow us to determine the inverse electronic compressibility and extract energy gaps of incompressible states. We observe integer and fractional Chern insulator states at superlattice filling factors v = 1, 2/3, and 1/3 with Streda slopes of -1, -2/3, and -1/3, respectively. Remarkably, the v = 1/3 state persists down to a magnetic field of 0.2 T. In addition, we also observe numerous trivial and topological charge density waves. We map out a phase diagram that is highly sensitive to both displacement and magnetic fields, which tune the system between various ground states by modifying the band dispersion and the structure of the electronic wavefunctions. This work demonstrates displacement field control of topological phase transitions in the moiré-proximal limit of rhombohedral pentalayer graphene, creating a highly-tunable platform for studying the interplay between intrinsic band topology and strong lattice effects. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Extended Quantum Anomalous Hall States in Graphene/hBN Moiré Superlattices

Authors: Zhengguang Lu, Tonghang Han, Yuxuan Yao, Jixiang Yang, Junseok Seo, Lihan Shi, Shenyong Ye, Kenji Watanabe, Takashi Taniguchi, Long Ju

Abstract: Electrons in topological flat bands can form novel topological states driven by the correlation effects. The penta-layer rhombohedral graphene/hBN moire superlattice has been shown to host fractional quantum anomalous Hall effect (FQAHE) at ~400 mK, triggering discussions around the underlying mechanism and the role of moire effects. In particular, novel electron crystal states with non-trivial to… ▽ More Electrons in topological flat bands can form novel topological states driven by the correlation effects. The penta-layer rhombohedral graphene/hBN moire superlattice has been shown to host fractional quantum anomalous Hall effect (FQAHE) at ~400 mK, triggering discussions around the underlying mechanism and the role of moire effects. In particular, novel electron crystal states with non-trivial topology have been proposed. Here we report DC electrical transport measurement in rhombohedral penta- and tetra-layer graphene/hBN moire superlattices at electronic temperatures down to ~40 mK. We observed two more FQAH states in the penta-layer devices than previously reported. In a new tetra-layer device, we observed FQAHE at filling factors v = 3/5 and 2/3 at 300 mK. With a small bias current and the lowest temperature, we observed a new extended quantum anomalous Hall (EQAH) state and magnetic hysteresis, where Rxy = h/e2 and vanishing Rxx span a wide range of moire filling factor v from 0.5 to up to 1.3. By increasing the temperature or current, FQAHE can be recovered -- suggesting the break-down of the EQAH states and a phase transition into the fractional quantum Hall liquid. Furthermore, we observed displacement field-induced quantum phase transitions from the EQAH states to Fermi liquid, FQAH liquid and the likely composite Fermi liquid. Our observation establishes a new topological phase of electrons with quantized Hall resistance at zero magnetic field, and enriches the emergent quantum phenomena in materials with topological flat bands. △ Less

Submitted 19 August, 2024; originally announced August 2024.

Diverse Impacts of Spin-Orbit Coupling on Superconductivity in Rhombohedral Graphene

Authors: Jixiang Yang, Xiaoyan Shi, Shenyong Ye, Chiho Yoon, Zhengguang Lu, Vivek Kakani, Tonghang Han, Junseok Seo, Lihan Shi, Kenji Watanabe, Takashi Taniguchi, Fan Zhang, Long Ju

Abstract: Engineering non-Abelian quasiparticles by combining superconductivity and topological states have been proposed as a route to realize topological quantum computation. Rhombohedral multilayer graphene with layer number N>=3 has been shown as a promising platform, as it hosts integer and fractional quantum anomalous Hall effects when proximitized by transition metal dichalcogenide (TMD) and a moire… ▽ More Engineering non-Abelian quasiparticles by combining superconductivity and topological states have been proposed as a route to realize topological quantum computation. Rhombohedral multilayer graphene with layer number N>=3 has been shown as a promising platform, as it hosts integer and fractional quantum anomalous Hall effects when proximitized by transition metal dichalcogenide (TMD) and a moire potential. However, superconductivity in similar devices have remained largely unexplored, although proximitized spin-orbit-coupling (SOC) effect has been shown to strengthen or induce superconductivity in both crystalline and twisted graphene. Here we report electron transport measurements of TMD-proximitized rhombohedral trilayer graphene (RTG) at temperatures down to 40 mK. We observed a new hole-doped superconducting state SC4 with a transition temperature Tc of 230 mK. On the electron-doped side, we identified a new isospin-symmetry breaking three-quarter-metal (TQM) phase. Near this three-quarter-metal state, the state SC3, very weak in bare RTG, is fully developed into a superconducting state at 110 mK. By performing fermiology analysis based on the quantum oscillation measurement, we showed that the SC3 and SC4 states reside at the phase boundaries between different isospin-symmetry-breaking states. These observations are aligned with the existing understanding that SOC enhances graphene superconductivity. Surprisingly, the original superconducting state SC1 in bare RTG is strongly suppressed in the presence of TMD, and we cannot find it down to the base temperature of our measurement. Our observations form the basis of exploring superconductivity and non-Abelian quasiparticles in rhombohedral graphene devices, and provide experimental evidence that challenges the understanding of the impacts of SOC on graphene superconductivity. △ Less

Submitted 19 August, 2024; originally announced August 2024.

Comments: 35 pages; 4 figures, 1 table, 13 extended data figures;

Numerical analysis of a class of penalty discontinuous Galerkin methods for nonlocal diffusion problems

Authors: Qiang Du, Lili Ju, Jianfang Lu, Xiaochuan Tian

Abstract: In this paper, we consider a class of discontinuous Galerkin (DG) methods for one-dimensional nonlocal diffusion (ND) problems. The nonlocal models, which are integral equations, are widely used in describing many physical phenomena with long-range interactions. The ND problem is the nonlocal analog of the classic diffusion problem, and as the interaction radius (horizon) vanishes, then the nonloc… ▽ More In this paper, we consider a class of discontinuous Galerkin (DG) methods for one-dimensional nonlocal diffusion (ND) problems. The nonlocal models, which are integral equations, are widely used in describing many physical phenomena with long-range interactions. The ND problem is the nonlocal analog of the classic diffusion problem, and as the interaction radius (horizon) vanishes, then the nonlocality disappears and the ND problem converges to the classic diffusion problem. Under certain conditions, the exact solution to the ND problem may exhibit discontinuities, setting it apart from the classic diffusion problem. Since the DG method shows its great advantages in resolving problems with discontinuities in computational fluid dynamics over the past several decades, it is natural to adopt the DG method to compute the ND problems. Based on [Du-Ju-Lu-Tian-CAMC2020], we develop the DG methods with different penalty terms, ensuring that the proposed DG methods have local counterparts as the horizon vanishes. This indicates the proposed methods will converge to the existing DG schemes as the horizon vanishes, which is crucial for achieving asymptotic compatibility. Rigorous proofs are provided to demonstrate the stability, error estimates, and asymptotic compatibility of the proposed DG schemes. To observe the effect of the nonlocal diffusion, we also consider the time-dependent convection-diffusion problems with nonlocal diffusion. We conduct several numerical experiments, including accuracy tests and Burgers' equation with nonlocal diffusion, and various horizons are taken to show the good performance of the proposed algorithm and validate the theoretical findings. △ Less

Submitted 13 August, 2024; originally announced August 2024.

MSC Class: 65M60; 65R20; 45A05

Swift-BAT GUANO follow-up of gravitational-wave triggers in the third LIGO-Virgo-KAGRA observing run

Authors: Gayathri Raman, Samuele Ronchini, James Delaunay, Aaron Tohuvavohu, Jamie A. Kennea, Tyler Parsotan, Elena Ambrosi, Maria Grazia Bernardini, Sergio Campana, Giancarlo Cusumano, Antonino D'Ai, Paolo D'Avanzo, Valerio D'Elia, Massimiliano De Pasquale, Simone Dichiara, Phil Evans, Dieter Hartmann, Paul Kuin, Andrea Melandri, Paul O'Brien, Julian P. Osborne, Kim Page, David M. Palmer, Boris Sbarufatti, Gianpiero Tagliaferri , et al. (1797 additional authors not shown)

Abstract: We present results from a search for X-ray/gamma-ray counterparts of gravitational-wave (GW) candidates from the third observing run (O3) of the LIGO-Virgo-KAGRA (LVK) network using the Swift Burst Alert Telescope (Swift-BAT). The search includes 636 GW candidates received in low latency, 86 of which have been confirmed by the offline analysis and included in the third cumulative Gravitational-Wav… ▽ More We present results from a search for X-ray/gamma-ray counterparts of gravitational-wave (GW) candidates from the third observing run (O3) of the LIGO-Virgo-KAGRA (LVK) network using the Swift Burst Alert Telescope (Swift-BAT). The search includes 636 GW candidates received in low latency, 86 of which have been confirmed by the offline analysis and included in the third cumulative Gravitational-Wave Transient Catalogs (GWTC-3). Targeted searches were carried out on the entire GW sample using the maximum--likelihood NITRATES pipeline on the BAT data made available via the GUANO infrastructure. We do not detect any significant electromagnetic emission that is temporally and spatially coincident with any of the GW candidates. We report flux upper limits in the 15-350 keV band as a function of sky position for all the catalog candidates. For GW candidates where the Swift-BAT false alarm rate is less than 10$^{-3}$ Hz, we compute the GW--BAT joint false alarm rate. Finally, the derived Swift-BAT upper limits are used to infer constraints on the putative electromagnetic emission associated with binary black hole mergers. △ Less

Submitted 13 July, 2024; originally announced July 2024.

Comments: 50 pages, 10 figures, 4 tables

TP-DRSeg: Improving Diabetic Retinopathy Lesion Segmentation with Explicit Text-Prompts Assisted SAM

Authors: Wenxue Li, Xinyu Xiong, Peng Xia, Lie Ju, Zongyuan Ge

Abstract: Recent advances in large foundation models, such as the Segment Anything Model (SAM), have demonstrated considerable promise across various tasks. Despite their progress, these models still encounter challenges in specialized medical image analysis, especially in recognizing subtle inter-class differences in Diabetic Retinopathy (DR) lesion segmentation. In this paper, we propose a novel framework… ▽ More Recent advances in large foundation models, such as the Segment Anything Model (SAM), have demonstrated considerable promise across various tasks. Despite their progress, these models still encounter challenges in specialized medical image analysis, especially in recognizing subtle inter-class differences in Diabetic Retinopathy (DR) lesion segmentation. In this paper, we propose a novel framework that customizes SAM for text-prompted DR lesion segmentation, termed TP-DRSeg. Our core idea involves exploiting language cues to inject medical prior knowledge into the vision-only segmentation network, thereby combining the advantages of different foundation models and enhancing the credibility of segmentation. Specifically, to unleash the potential of vision-language models in the recognition of medical concepts, we propose an explicit prior encoder that transfers implicit medical concepts into explicit prior knowledge, providing explainable clues to excavate low-level features associated with lesions. Furthermore, we design a prior-aligned injector to inject explicit priors into the segmentation process, which can facilitate knowledge sharing across multi-modality features and allow our framework to be trained in a parameter-efficient fashion. Experimental results demonstrate the superiority of our framework over other traditional models and foundation model variants. △ Less

Submitted 22 June, 2024; originally announced June 2024.

Demonstration of optical spring in an un-detuned cavity containing an optical parametric amplifier

Authors: Jian Liu, Juntao Pan, Carl Blair, Jue Zhang, Hengxin Sun, Li Ju, Chunnong Zhao

Abstract: Here we demonstrate the capacity to manipulate the optical spring (OS) effect by employing an optical parametric amplifier (OPA) within an optical cavity. We observed more than a factor of 2 increase in the OS frequency shift with the OPA. We also showed for the first time that the OS can be tuned by solely adjusting the OPA phase and showing an un-detuned cavity exhibiting an optical spring. The… ▽ More Here we demonstrate the capacity to manipulate the optical spring (OS) effect by employing an optical parametric amplifier (OPA) within an optical cavity. We observed more than a factor of 2 increase in the OS frequency shift with the OPA. We also showed for the first time that the OS can be tuned by solely adjusting the OPA phase and showing an un-detuned cavity exhibiting an optical spring. The method can be applied to gravitational wave detectors in the signal recycling configuration to realize narrow bandwidth high sensitivity. The OS can be tuned to align the detector peak sensitivity frequency to known frequency continuous gravitational wave signals, dynamically tuned to track the gravitational wave signal from merging compact binaries or tuned to search for the post-merger signal of known binary coalescence. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Comments: 6 pages, 9 figures

Generalizing to Unseen Domains in Diabetic Retinopathy with Disentangled Representations

Authors: Peng Xia, Ming Hu, Feilong Tang, Wenxue Li, Wenhao Zheng, Lie Ju, Peibo Duan, Huaxiu Yao, Zongyuan Ge

Abstract: Diabetic Retinopathy (DR), induced by diabetes, poses a significant risk of visual impairment. Accurate and effective grading of DR aids in the treatment of this condition. Yet existing models experience notable performance degradation on unseen domains due to domain shifts. Previous methods address this issue by simulating domain style through simple visual transformation and mitigating domain no… ▽ More Diabetic Retinopathy (DR), induced by diabetes, poses a significant risk of visual impairment. Accurate and effective grading of DR aids in the treatment of this condition. Yet existing models experience notable performance degradation on unseen domains due to domain shifts. Previous methods address this issue by simulating domain style through simple visual transformation and mitigating domain noise via learning robust representations. However, domain shifts encompass more than image styles. They overlook biases caused by implicit factors such as ethnicity, age, and diagnostic criteria. In our work, we propose a novel framework where representations of paired data from different domains are decoupled into semantic features and domain noise. The resulting augmented representation comprises original retinal semantics and domain noise from other domains, aiming to generate enhanced representations aligned with real-world clinical needs, incorporating rich information from diverse domains. Subsequently, to improve the robustness of the decoupled representations, class and domain prototypes are employed to interpolate the disentangled representations while data-aware weights are designed to focus on rare classes and domains. Finally, we devise a robust pixel-level semantic alignment loss to align retinal semantics decoupled from features, maintaining a balance between intra-class diversity and dense class features. Experimental results on multiple benchmarks demonstrate the effectiveness of our method on unseen domains. The code implementations are accessible on https://github.com/richard-peng-xia/DECO. △ Less

Submitted 10 June, 2024; originally announced June 2024.

Comments: Early Accepted by MICCAI 2024

Similarity is Not All You Need: Endowing Retrieval Augmented Generation with Multi Layered Thoughts

Authors: Chunjing Gan, Dan Yang, Binbin Hu, Hanxiao Zhang, Siyuan Li, Ziqi Liu, Yue Shen, Lin Ju, Zhiqiang Zhang, Jinjie Gu, Lei Liang, Jun Zhou

Abstract: In recent years, large language models (LLMs) have made remarkable achievements in various domains. However, the untimeliness and cost of knowledge updates coupled with hallucination issues of LLMs have curtailed their applications in knowledge intensive tasks, where retrieval augmented generation (RAG) can be of help. Nevertheless, existing retrieval augmented models typically use similarity as a… ▽ More In recent years, large language models (LLMs) have made remarkable achievements in various domains. However, the untimeliness and cost of knowledge updates coupled with hallucination issues of LLMs have curtailed their applications in knowledge intensive tasks, where retrieval augmented generation (RAG) can be of help. Nevertheless, existing retrieval augmented models typically use similarity as a bridge between queries and documents and follow a retrieve then read procedure. In this work, we argue that similarity is not always the panacea and totally relying on similarity would sometimes degrade the performance of retrieval augmented generation. To this end, we propose MetRag, a Multi layEred Thoughts enhanced Retrieval Augmented Generation framework. To begin with, beyond existing similarity oriented thought, we embrace a small scale utility model that draws supervision from an LLM for utility oriented thought and further come up with a smarter model by comprehensively combining the similarity and utility oriented thoughts. Furthermore, given the fact that the retrieved document set tends to be huge and using them in isolation makes it difficult to capture the commonalities and characteristics among them, we propose to make an LLM as a task adaptive summarizer to endow retrieval augmented generation with compactness-oriented thought. Finally, with multi layered thoughts from the precedent stages, an LLM is called for knowledge augmented generation. Extensive experiments on knowledge-intensive tasks have demonstrated the superiority of MetRag. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Comments: 12 pages

EfficientGS: Streamlining Gaussian Splatting for Large-Scale High-Resolution Scene Representation

Authors: Wenkai Liu, Tao Guan, Bin Zhu, Lili Ju, Zikai Song, Dan Li, Yuesong Wang, Wei Yang

Abstract: In the domain of 3D scene representation, 3D Gaussian Splatting (3DGS) has emerged as a pivotal technology. However, its application to large-scale, high-resolution scenes (exceeding 4k$\times$4k pixels) is hindered by the excessive computational requirements for managing a large number of Gaussians. Addressing this, we introduce 'EfficientGS', an advanced approach that optimizes 3DGS for high-res… ▽ More In the domain of 3D scene representation, 3D Gaussian Splatting (3DGS) has emerged as a pivotal technology. However, its application to large-scale, high-resolution scenes (exceeding 4k$\times$4k pixels) is hindered by the excessive computational requirements for managing a large number of Gaussians. Addressing this, we introduce 'EfficientGS', an advanced approach that optimizes 3DGS for high-resolution, large-scale scenes. We analyze the densification process in 3DGS and identify areas of Gaussian over-proliferation. We propose a selective strategy, limiting Gaussian increase to key primitives, thereby enhancing the representational efficiency. Additionally, we develop a pruning mechanism to remove redundant Gaussians, those that are merely auxiliary to adjacent ones. For further enhancement, we integrate a sparse order increment for Spherical Harmonics (SH), designed to alleviate storage constraints and reduce training overhead. Our empirical evaluations, conducted on a range of datasets including extensive 4K+ aerial images, demonstrate that 'EfficientGS' not only expedites training and rendering times but also achieves this with a model size approximately tenfold smaller than conventional 3DGS while maintaining high rendering fidelity. △ Less

Submitted 19 April, 2024; originally announced April 2024.

AntBatchInfer: Elastic Batch Inference in the Kubernetes Cluster

Authors: Siyuan Li, Youshao Xiao, Fanzhuang Meng, Lin Ju, Lei Liang, Lin Wang, Jun Zhou

Abstract: Offline batch inference is a common task in the industry for deep learning applications, but it can be challenging to ensure stability and performance when dealing with large amounts of data and complicated inference pipelines. This paper demonstrated AntBatchInfer, an elastic batch inference framework, which is specially optimized for the non-dedicated cluster. AntBatchInfer addresses these chall… ▽ More Offline batch inference is a common task in the industry for deep learning applications, but it can be challenging to ensure stability and performance when dealing with large amounts of data and complicated inference pipelines. This paper demonstrated AntBatchInfer, an elastic batch inference framework, which is specially optimized for the non-dedicated cluster. AntBatchInfer addresses these challenges by providing multi-level fault-tolerant capabilities, enabling the stable execution of versatile and long-running inference tasks. It also improves inference efficiency by pipelining, intra-node, and inter-node scaling. It further optimizes the performance in complicated multiple-model batch inference scenarios. Through extensive experiments and real-world statistics, we demonstrate the superiority of our framework in terms of stability and efficiency. In the experiment, it outperforms the baseline by at least $2\times$ and $6\times$ in the single-model or multiple-model batch inference. Also, it is widely used at Ant Group, with thousands of daily jobs from various scenarios, including DLRM, CV, and NLP, which proves its practicability in the industry. △ Less

Submitted 15 April, 2024; originally announced April 2024.

AntDT: A Self-Adaptive Distributed Training Framework for Leader and Straggler Nodes

Authors: Youshao Xiao, Lin Ju, Zhenglei Zhou, Siyuan Li, Zhaoxin Huan, Dalong Zhang, Rujie Jiang, Lin Wang, Xiaolu Zhang, Lei Liang, Jun Zhou

Abstract: Many distributed training techniques like Parameter Server and AllReduce have been proposed to take advantage of the increasingly large data and rich features. However, stragglers frequently occur in distributed training due to resource contention and hardware heterogeneity, which significantly hampers the training efficiency. Previous works only address part of the stragglers and could not adapti… ▽ More Many distributed training techniques like Parameter Server and AllReduce have been proposed to take advantage of the increasingly large data and rich features. However, stragglers frequently occur in distributed training due to resource contention and hardware heterogeneity, which significantly hampers the training efficiency. Previous works only address part of the stragglers and could not adaptively solve various stragglers in practice. Additionally, it is challenging to use a systematic framework to address all stragglers because different stragglers require diverse data allocation and fault-tolerance mechanisms. Therefore, this paper proposes a unified distributed training framework called AntDT (Ant Distributed Training Framework) to adaptively solve the straggler problems. Firstly, the framework consists of four components, including the Stateful Dynamic Data Sharding service, Monitor, Controller, and Agent. These components work collaboratively to efficiently distribute workloads and provide a range of pre-defined straggler mitigation methods with fault tolerance, thereby hiding messy details of data allocation and fault handling. Secondly, the framework provides a high degree of flexibility, allowing for the customization of straggler mitigation solutions based on the specific circumstances of the cluster. Leveraging this flexibility, we introduce two straggler mitigation solutions, namely AntDT-ND for non-dedicated clusters and AntDT-DD for dedicated clusters, as practical examples to resolve various types of stragglers at Ant Group. Justified by our comprehensive experiments and industrial deployment statistics, AntDT outperforms other SOTA methods more than 3x in terms of training efficiency. Additionally, in Alipay's homepage recommendation scenario, using AntDT reduces the training duration of the ranking model from 27.8 hours to just 5.4 hours. △ Less

Submitted 15 April, 2024; originally announced April 2024.

Observation of Gravitational Waves from the Coalescence of a $2.5\text{-}4.5~M_\odot$ Compact Object and a Neutron Star

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, A. G. Abac, R. Abbott, I. Abouelfettouh, F. Acernese, K. Ackley, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, D. Agarwal, M. Agathos, M. Aghaei Abchouyeh, O. D. Aguiar, I. Aguilar, L. Aiello, A. Ain, P. Ajith, S. Akçay, T. Akutsu, S. Albanesi, R. A. Alfaidi, A. Al-Jodah , et al. (1771 additional authors not shown)

Abstract: We report the observation of a coalescing compact binary with component masses $2.5\text{-}4.5~M_\odot$ and $1.2\text{-}2.0~M_\odot$ (all measurements quoted at the 90% credible level). The gravitational-wave signal GW230529_181500 was observed during the fourth observing run of the LIGO-Virgo-KAGRA detector network on 2023 May 29 by the LIGO Livingston Observatory. The primary component of the so… ▽ More We report the observation of a coalescing compact binary with component masses $2.5\text{-}4.5~M_\odot$ and $1.2\text{-}2.0~M_\odot$ (all measurements quoted at the 90% credible level). The gravitational-wave signal GW230529_181500 was observed during the fourth observing run of the LIGO-Virgo-KAGRA detector network on 2023 May 29 by the LIGO Livingston Observatory. The primary component of the source has a mass less than $5~M_\odot$ at 99% credibility. We cannot definitively determine from gravitational-wave data alone whether either component of the source is a neutron star or a black hole. However, given existing estimates of the maximum neutron star mass, we find the most probable interpretation of the source to be the coalescence of a neutron star with a black hole that has a mass between the most massive neutron stars and the least massive black holes observed in the Galaxy. We provisionally estimate a merger rate density of $55^{+127}_{-47}~\text{Gpc}^{-3}\,\text{yr}^{-1}$ for compact binary coalescences with properties similar to the source of GW230529_181500; assuming that the source is a neutron star-black hole merger, GW230529_181500-like sources constitute about 60% of the total merger rate inferred for neutron star-black hole coalescences. The discovery of this system implies an increase in the expected rate of neutron star-black hole mergers with electromagnetic counterparts and provides further evidence for compact objects existing within the purported lower mass gap. △ Less

Submitted 26 July, 2024; v1 submitted 5 April, 2024; originally announced April 2024.

Comments: 45 pages (10 pages author list, 13 pages main text, 1 page acknowledgements, 13 pages appendices, 8 pages bibliography), 17 figures, 16 tables. Update to match version published in The Astrophysical Journal Letters. Data products available from https://zenodo.org/records/10845779

Report number: LIGO-P2300352

Journal ref: ApJL 970, L34 (2024)

Diversified and Personalized Multi-rater Medical Image Segmentation

Authors: Yicheng Wu, Xiangde Luo, Zhe Xu, Xiaoqing Guo, Lie Ju, Zongyuan Ge, Wenjun Liao, Jianfei Cai

Abstract: Annotation ambiguity due to inherent data uncertainties such as blurred boundaries in medical scans and different observer expertise and preferences has become a major obstacle for training deep-learning based medical image segmentation models. To address it, the common practice is to gather multiple annotations from different experts, leading to the setting of multi-rater medical image segmentati… ▽ More Annotation ambiguity due to inherent data uncertainties such as blurred boundaries in medical scans and different observer expertise and preferences has become a major obstacle for training deep-learning based medical image segmentation models. To address it, the common practice is to gather multiple annotations from different experts, leading to the setting of multi-rater medical image segmentation. Existing works aim to either merge different annotations into the "groundtruth" that is often unattainable in numerous medical contexts, or generate diverse results, or produce personalized results corresponding to individual expert raters. Here, we bring up a more ambitious goal for multi-rater medical image segmentation, i.e., obtaining both diversified and personalized results. Specifically, we propose a two-stage framework named D-Persona (first Diversification and then Personalization). In Stage I, we exploit multiple given annotations to train a Probabilistic U-Net model, with a bound-constrained loss to improve the prediction diversity. In this way, a common latent space is constructed in Stage I, where different latent codes denote diversified expert opinions. Then, in Stage II, we design multiple attention-based projection heads to adaptively query the corresponding expert prompts from the shared latent space, and then perform the personalized medical image segmentation. We evaluated the proposed model on our in-house Nasopharyngeal Carcinoma dataset and the public lung nodule dataset (i.e., LIDC-IDRI). Extensive experiments demonstrated our D-Persona can provide diversified and personalized results at the same time, achieving new SOTA performance for multi-rater medical image segmentation. Our code will be released at https://github.com/ycwu1997/D-Persona. △ Less

Submitted 20 March, 2024; originally announced March 2024.

Comments: Accepted by CVPR 2024

Ultralight vector dark matter search using data from the KAGRA O3GK run

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, A. G. Abac, R. Abbott, H. Abe, I. Abouelfettouh, F. Acernese, K. Ackley, C. Adamcewicz, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, O. D. Aguiar, I. Aguilar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi , et al. (1778 additional authors not shown)

Abstract: Among the various candidates for dark matter (DM), ultralight vector DM can be probed by laser interferometric gravitational wave detectors through the measurement of oscillating length changes in the arm cavities. In this context, KAGRA has a unique feature due to differing compositions of its mirrors, enhancing the signal of vector DM in the length change in the auxiliary channels. Here we prese… ▽ More Among the various candidates for dark matter (DM), ultralight vector DM can be probed by laser interferometric gravitational wave detectors through the measurement of oscillating length changes in the arm cavities. In this context, KAGRA has a unique feature due to differing compositions of its mirrors, enhancing the signal of vector DM in the length change in the auxiliary channels. Here we present the result of a search for $U(1)_{B-L}$ gauge boson DM using the KAGRA data from auxiliary length channels during the first joint observation run together with GEO600. By applying our search pipeline, which takes into account the stochastic nature of ultralight DM, upper bounds on the coupling strength between the $U(1)_{B-L}$ gauge boson and ordinary matter are obtained for a range of DM masses. While our constraints are less stringent than those derived from previous experiments, this study demonstrates the applicability of our method to the lower-mass vector DM search, which is made difficult in this measurement by the short observation time compared to the auto-correlation time scale of DM. △ Less

Submitted 5 March, 2024; originally announced March 2024.

Comments: 20 pages, 5 figures

Report number: LIGO-P2300250

A Riemann-Hilbert approach to the two-component modified Camassa-Holm equation

Authors: Kai Xu, Luman Ju, Engui Fan

Abstract: In this paper, we develop a Riemann-Hilbert (RH) approach to the Cauchy problem for the two-component modified Camassa-Holm (2-mCH) equation based on its Lax pair. Further via a series of deformations to the RH problem by using the $\bar{\partial}$-generalization of Deift-Zhou steepest descent method, we obtain the long-time asymptotic approximations to the solutions of the 2-mCH equation in four… ▽ More In this paper, we develop a Riemann-Hilbert (RH) approach to the Cauchy problem for the two-component modified Camassa-Holm (2-mCH) equation based on its Lax pair. Further via a series of deformations to the RH problem by using the $\bar{\partial}$-generalization of Deift-Zhou steepest descent method, we obtain the long-time asymptotic approximations to the solutions of the 2-mCH equation in four kinds of space-time regions. Especially we introduce a technique to unify multi-jump matrix factorizations into one form which can greatly simplify the calculation of the $\bar{\partial}$-steepest descent method. △ Less

Submitted 26 August, 2024; v1 submitted 21 February, 2024; originally announced February 2024.

Comments: 27 pages

MSC Class: 35Q51; 35Q15; 37K15; 35C20

SGV3D:Towards Scenario Generalization for Vision-based Roadside 3D Object Detection

Authors: Lei Yang, Xinyu Zhang, Jun Li, Li Wang, Chuang Zhang, Li Ju, Zhiwei Li, Yang Shen

Abstract: Roadside perception can greatly increase the safety of autonomous vehicles by extending their perception ability beyond the visual range and addressing blind spots. However, current state-of-the-art vision-based roadside detection methods possess high accuracy on labeled scenes but have inferior performance on new scenes. This is because roadside cameras remain stationary after installation and ca… ▽ More Roadside perception can greatly increase the safety of autonomous vehicles by extending their perception ability beyond the visual range and addressing blind spots. However, current state-of-the-art vision-based roadside detection methods possess high accuracy on labeled scenes but have inferior performance on new scenes. This is because roadside cameras remain stationary after installation and can only collect data from a single scene, resulting in the algorithm overfitting these roadside backgrounds and camera poses. To address this issue, in this paper, we propose an innovative Scenario Generalization Framework for Vision-based Roadside 3D Object Detection, dubbed SGV3D. Specifically, we employ a Background-suppressed Module (BSM) to mitigate background overfitting in vision-centric pipelines by attenuating background features during the 2D to bird's-eye-view projection. Furthermore, by introducing the Semi-supervised Data Generation Pipeline (SSDG) using unlabeled images from new scenes, diverse instance foregrounds with varying camera poses are generated, addressing the risk of overfitting specific camera poses. We evaluate our method on two large-scale roadside benchmarks. Our method surpasses all previous methods by a significant margin in new scenes, including +42.57% for vehicle, +5.87% for pedestrian, and +14.89% for cyclist compared to BEVHeight on the DAIR-V2X-I heterologous benchmark. On the larger-scale Rope3D heterologous benchmark, we achieve notable gains of 14.48% for car and 12.41% for large vehicle. We aspire to contribute insights on the exploration of roadside perception techniques, emphasizing their capability for scenario generalization. The code will be available at https://github.com/yanglei18/SGV3D △ Less

Submitted 9 April, 2024; v1 submitted 29 January, 2024; originally announced January 2024.

Comments: 13 pages, 8 figures

M2-Encoder: Advancing Bilingual Image-Text Understanding by Large-scale Efficient Pretraining

Authors: Qingpei Guo, Furong Xu, Hanxiao Zhang, Wang Ren, Ziping Ma, Lin Ju, Jian Wang, Jingdong Chen, Ming Yang

Abstract: Vision-language foundation models like CLIP have revolutionized the field of artificial intelligence. Nevertheless, VLM models supporting multi-language, e.g., in both Chinese and English, have lagged due to the relative scarcity of large-scale pretraining datasets. Toward this end, we introduce a comprehensive bilingual (Chinese-English) dataset BM-6B with over 6 billion image-text pairs, aimed a… ▽ More Vision-language foundation models like CLIP have revolutionized the field of artificial intelligence. Nevertheless, VLM models supporting multi-language, e.g., in both Chinese and English, have lagged due to the relative scarcity of large-scale pretraining datasets. Toward this end, we introduce a comprehensive bilingual (Chinese-English) dataset BM-6B with over 6 billion image-text pairs, aimed at enhancing multimodal foundation models to well understand images in both languages. To handle such a scale of dataset, we propose a novel grouped aggregation approach for image-text contrastive loss computation, which reduces the communication overhead and GPU memory demands significantly, facilitating a 60% increase in training speed. We pretrain a series of bilingual image-text foundation models with an enhanced fine-grained understanding ability on BM-6B, the resulting models, dubbed as $M^2$-Encoders (pronounced "M-Square"), set new benchmarks in both languages for multimodal retrieval and classification tasks. Notably, Our largest $M^2$-Encoder-10B model has achieved top-1 accuracies of 88.5% on ImageNet and 80.7% on ImageNet-CN under a zero-shot classification setting, surpassing previously reported SoTA methods by 2.2% and 21.1%, respectively. The $M^2$-Encoder series represents one of the most comprehensive bilingual image-text foundation models to date, so we are making it available to the research community for further exploration and development. △ Less

Submitted 3 February, 2024; v1 submitted 29 January, 2024; originally announced January 2024.

Gradient Preserving Operator Inference: Data-Driven Reduced-Order Models for Equations with Gradient Structure

Authors: Yuwei Geng, Jasdeep Singh, Lili Ju, Boris Kramer, Zhu Wang

Abstract: Hamiltonian Operator Inference has been introduced in [Sharma, H., Wang, Z., Kramer, B., Physica D: Nonlinear Phenomena, 431, p.133122, 2022] to learn structure-preserving reduced-order models (ROMs) for Hamiltonian systems. This approach constructs a low-dimensional model using only data and knowledge of the Hamiltonian function. Such ROMs can keep the intrinsic structure of the system, allowing… ▽ More Hamiltonian Operator Inference has been introduced in [Sharma, H., Wang, Z., Kramer, B., Physica D: Nonlinear Phenomena, 431, p.133122, 2022] to learn structure-preserving reduced-order models (ROMs) for Hamiltonian systems. This approach constructs a low-dimensional model using only data and knowledge of the Hamiltonian function. Such ROMs can keep the intrinsic structure of the system, allowing them to capture the physics described by the governing equations. In this work, we extend this approach to more general systems that are either conservative or dissipative in energy, and which possess a gradient structure. We derive the optimization problems for inferring structure-preserving ROMs that preserve the gradient structure. We further derive an $a\ priori$ error estimate for the reduced-order approximation. To test the algorithms, we consider semi-discretized partial differential equations with gradient structure, such as the parameterized wave and Korteweg-de-Vries equations, and equations of three-dimensional linear elasticity in the conservative case and the one- and two-dimensional Allen-Cahn equations in the dissipative case. The numerical results illustrate the accuracy, structure-preservation properties, and predictive capabilities of the gradient-preserving Operator Inference ROMs. △ Less

Submitted 9 May, 2024; v1 submitted 22 January, 2024; originally announced January 2024.

Comments: 36 pages

MSC Class: 65P99; 65M15

Journal ref: Computer Methods in Applied Mechanics and Engineering, Volume 427, 2024, 117033

A novel bond-based nonlocal diffusion model with matrix-valued coefficients in non-divergence form and its collocation discretization

Authors: Lili Ju, Hao Tian, Junke Lu

Abstract: Existing nonlocal diffusion models are predominantly classified into two categories: bond-based models, which involve a single-fold integral and usually simulate isotropic diffusion, and state-based models, which contain a double-fold integral and can additionally prototype anisotropic diffusion. While bond-based models exhibit computational efficiency, they are somewhat limited in their modeling… ▽ More Existing nonlocal diffusion models are predominantly classified into two categories: bond-based models, which involve a single-fold integral and usually simulate isotropic diffusion, and state-based models, which contain a double-fold integral and can additionally prototype anisotropic diffusion. While bond-based models exhibit computational efficiency, they are somewhat limited in their modeling capabilities. In this paper, we develop a novel bond-based nonlocal diffusion model with matrix-valued coefficients in non-divergence form. Our approach incorporates the coefficients into a covariance matrix and employs the multivariate Gaussian function with truncation to define the kernel function, and subsequently model the nonlocal diffusion process through the bond-based formulation. We successfully establish the well-posedness of the proposed model along with deriving some of its properties on maximum principle and mass conservation. Furthermore, an efficient linear collocation scheme is designed for numerical solution of our model. Comprehensive experiments in two and three dimensions are conducted to showcase application of the proposed nonlocal model to both isotropic and anisotropic diffusion problems and to demonstrate numerical accuracy and effective asymptotic compatibility of the proposed collocation scheme. △ Less

Submitted 10 January, 2024; originally announced January 2024.

A high-order multi-time-step scheme for bond-based peridynamics

Authors: Chenguang Liu, Jie Sun, Hao Tian, WaiSun Don, Lili Ju

Abstract: A high-order multi-time-step (MTS) scheme for the bond-based peridynamic (PD) model, an extension of classical continuous mechanics widely used for analyzing discontinuous problems like cracks, is proposed. The MTS scheme discretizes the spatial domain with a meshfree method and advances in time with a high-order Runge-Kutta method. To effectively handle discontinuities (cracks) that appear in a l… ▽ More A high-order multi-time-step (MTS) scheme for the bond-based peridynamic (PD) model, an extension of classical continuous mechanics widely used for analyzing discontinuous problems like cracks, is proposed. The MTS scheme discretizes the spatial domain with a meshfree method and advances in time with a high-order Runge-Kutta method. To effectively handle discontinuities (cracks) that appear in a local subdomain in the solution, the scheme employs the Taylor expansion and Lagrange interpolation polynomials with a finer time step size, that is, coarse and fine time step sizes for smooth and discontinuous subdomains, respectively, to achieve accurate and efficient simulations. By eliminating unnecessary fine-scale resolution imposed on the entire domain, the MTS scheme outperforms the standard PD scheme by significantly reducing computational costs, particularly for problems with discontinuous solutions, as demonstrated by comprehensive theoretical analysis and numerical experiments. △ Less

Submitted 10 January, 2024; originally announced January 2024.

G-Meta: Distributed Meta Learning in GPU Clusters for Large-Scale Recommender Systems

Authors: Youshao Xiao, Shangchun Zhao, Zhenglei Zhou, Zhaoxin Huan, Lin Ju, Xiaolu Zhang, Lin Wang, Jun Zhou

Abstract: Recently, a new paradigm, meta learning, has been widely applied to Deep Learning Recommendation Models (DLRM) and significantly improves statistical performance, especially in cold-start scenarios. However, the existing systems are not tailored for meta learning based DLRM models and have critical problems regarding efficiency in distributed training in the GPU cluster. It is because the conventi… ▽ More Recently, a new paradigm, meta learning, has been widely applied to Deep Learning Recommendation Models (DLRM) and significantly improves statistical performance, especially in cold-start scenarios. However, the existing systems are not tailored for meta learning based DLRM models and have critical problems regarding efficiency in distributed training in the GPU cluster. It is because the conventional deep learning pipeline is not optimized for two task-specific datasets and two update loops in meta learning. This paper provides a high-performance framework for large-scale training for Optimization-based Meta DLRM models over the \textbf{G}PU cluster, namely \textbf{G}-Meta. Firstly, G-Meta utilizes both data parallelism and model parallelism with careful orchestration regarding computation and communication efficiency, to enable high-speed distributed training. Secondly, it proposes a Meta-IO pipeline for efficient data ingestion to alleviate the I/O bottleneck. Various experimental results show that G-Meta achieves notable training speed without loss of statistical performance. Since early 2022, G-Meta has been deployed in Alipay's core advertising and recommender system, shrinking the continuous delivery of models by four times. It also obtains 6.48\% improvement in Conversion Rate (CVR) and 1.06\% increase in CPM (Cost Per Mille) in Alipay's homepage display advertising, with the benefit of larger training samples and tasks. △ Less

Submitted 8 January, 2024; originally announced January 2024.

Prompt-driven Latent Domain Generalization for Medical Image Classification

Authors: Siyuan Yan, Chi Liu, Zhen Yu, Lie Ju, Dwarikanath Mahapatra, Brigid Betz-Stablein, Victoria Mar, Monika Janda, Peter Soyer, Zongyuan Ge

Abstract: Deep learning models for medical image analysis easily suffer from distribution shifts caused by dataset artifacts bias, camera variations, differences in the imaging station, etc., leading to unreliable diagnoses in real-world clinical settings. Domain generalization (DG) methods, which aim to train models on multiple domains to perform well on unseen domains, offer a promising direction to solve… ▽ More Deep learning models for medical image analysis easily suffer from distribution shifts caused by dataset artifacts bias, camera variations, differences in the imaging station, etc., leading to unreliable diagnoses in real-world clinical settings. Domain generalization (DG) methods, which aim to train models on multiple domains to perform well on unseen domains, offer a promising direction to solve the problem. However, existing DG methods assume domain labels of each image are available and accurate, which is typically feasible for only a limited number of medical datasets. To address these challenges, we propose a novel DG framework for medical image classification without relying on domain labels, called Prompt-driven Latent Domain Generalization (PLDG). PLDG consists of unsupervised domain discovery and prompt learning. This framework first discovers pseudo domain labels by clustering the bias-associated style features, then leverages collaborative domain prompts to guide a Vision Transformer to learn knowledge from discovered diverse domains. To facilitate cross-domain knowledge learning between different prompts, we introduce a domain prompt generator that enables knowledge sharing between domain prompts and a shared prompt. A domain mixup strategy is additionally employed for more flexible decision margins and mitigates the risk of incorrect domain assignments. Extensive experiments on three medical image classification tasks and one debiasing task demonstrate that our method can achieve comparable or even superior performance than conventional DG algorithms without relying on domain labels. Our code will be publicly available upon the paper is accepted. △ Less

Submitted 5 January, 2024; originally announced January 2024.

Comments: 10 pages

An Adaptive Placement and Parallelism Framework for Accelerating RLHF Training

Authors: Youshao Xiao, Weichang Wu, Zhenglei Zhou, Fagui Mao, Shangchun Zhao, Lin Ju, Lei Liang, Xiaolu Zhang, Jun Zhou

Abstract: Recently, ChatGPT or InstructGPT like large language models (LLM) has made a significant impact in the AI world. Many works have attempted to reproduce the complex InstructGPT's training pipeline, namely Reinforcement Learning with Human Feedback (RLHF). However, the mainstream distributed RLHF training methods typically adopt a fixed model placement strategy, referred to as the Flattening strateg… ▽ More Recently, ChatGPT or InstructGPT like large language models (LLM) has made a significant impact in the AI world. Many works have attempted to reproduce the complex InstructGPT's training pipeline, namely Reinforcement Learning with Human Feedback (RLHF). However, the mainstream distributed RLHF training methods typically adopt a fixed model placement strategy, referred to as the Flattening strategy. This strategy treats all four interdependent models involved in RLHF as a single entity, distributing them across all devices and applying parallelism techniques designed for a single model, regardless of the different workloads inherent to each model. As a result, this strategy exacerbates the generation bottlenecks in the RLHF training and degrades the overall training efficiency. To address these issues, we propose an adaptive model placement framework that offers two flexible model placement strategies. The Interleaving strategy helps reduce memory redundancy and communication costs of RLHF training by placing models without dependencies on exclusive devices with careful orchestration. On the other hand, the Separation strategy improves the throughput of model training by separating the training and inference runtime of the RLHF pipeline with additional shadow models. Furthermore, our framework provides a simple user interface and allows for the agile allocation of models across devices in a fine-grained manner for various training scenarios, involving models of varying sizes and devices of different scales. Extensive experiments have demonstrated that our Interleaving and Separation strategies can achieve notable improvements up to 11X, compared to the current SOTA approaches. The results highlight the effectiveness and adaptability of our approaches in accelerating the training of distributed RLHF. △ Less

Submitted 24 January, 2024; v1 submitted 18 December, 2023; originally announced December 2023.

AVA: Inconspicuous Attribute Variation-based Adversarial Attack bypassing DeepFake Detection

Authors: Xiangtao Meng, Li Wang, Shanqing Guo, Lei Ju, Qingchuan Zhao

Abstract: While DeepFake applications are becoming popular in recent years, their abuses pose a serious privacy threat. Unfortunately, most related detection algorithms to mitigate the abuse issues are inherently vulnerable to adversarial attacks because they are built atop DNN-based classification models, and the literature has demonstrated that they could be bypassed by introducing pixel-level perturbatio… ▽ More While DeepFake applications are becoming popular in recent years, their abuses pose a serious privacy threat. Unfortunately, most related detection algorithms to mitigate the abuse issues are inherently vulnerable to adversarial attacks because they are built atop DNN-based classification models, and the literature has demonstrated that they could be bypassed by introducing pixel-level perturbations. Though corresponding mitigation has been proposed, we have identified a new attribute-variation-based adversarial attack (AVA) that perturbs the latent space via a combination of Gaussian prior and semantic discriminator to bypass such mitigation. It perturbs the semantics in the attribute space of DeepFake images, which are inconspicuous to human beings (e.g., mouth open) but can result in substantial differences in DeepFake detection. We evaluate our proposed AVA attack on nine state-of-the-art DeepFake detection algorithms and applications. The empirical results demonstrate that AVA attack defeats the state-of-the-art black box attacks against DeepFake detectors and achieves more than a 95% success rate on two commercial DeepFake detectors. Moreover, our human study indicates that AVA-generated DeepFake images are often imperceptible to humans, which presents huge security and privacy concerns. △ Less

Submitted 14 December, 2023; originally announced December 2023.

A new approach for the implementation of contact line motion based on the phase-filed lattice Boltzmann method

Authors: Long Ju, Zhaoli Guo, Bicheng Yan, Shuyu Sun

Abstract: This paper proposes a new strategy to implement the free-energy based wetting boundary condition within the phase-field lattice Boltzmann method. The greatest advantage of the proposed method is that the implementation of contact line motion can be significantly simplified while still maintaining good accuracy. For this purpose, the liquid-solid free energy is treated as a part of the chemical pot… ▽ More This paper proposes a new strategy to implement the free-energy based wetting boundary condition within the phase-field lattice Boltzmann method. The greatest advantage of the proposed method is that the implementation of contact line motion can be significantly simplified while still maintaining good accuracy. For this purpose, the liquid-solid free energy is treated as a part of the chemical potential instead of the boundary condition, thus avoiding complicated interpolations with irregular geometries. Several numerical testing cases including the droplet spreading processes on the idea flat, inclined and curved boundaries are conducted, and the results demonstrate that the proposed method has good ability and satisfactory accuracy to simulate contact line motions. △ Less

Submitted 1 December, 2023; originally announced December 2023.

HGCLIP: Exploring Vision-Language Models with Graph Representations for Hierarchical Understanding

Authors: Peng Xia, Xingtong Yu, Ming Hu, Lie Ju, Zhiyong Wang, Peibo Duan, Zongyuan Ge

Abstract: Object categories are typically organized into a multi-granularity taxonomic hierarchy. When classifying categories at different hierarchy levels, traditional uni-modal approaches focus primarily on image features, revealing limitations in complex scenarios. Recent studies integrating Vision-Language Models (VLMs) with class hierarchies have shown promise, yet they fall short of fully exploiting t… ▽ More Object categories are typically organized into a multi-granularity taxonomic hierarchy. When classifying categories at different hierarchy levels, traditional uni-modal approaches focus primarily on image features, revealing limitations in complex scenarios. Recent studies integrating Vision-Language Models (VLMs) with class hierarchies have shown promise, yet they fall short of fully exploiting the hierarchical relationships. These efforts are constrained by their inability to perform effectively across varied granularity of categories. To tackle this issue, we propose a novel framework (HGCLIP) that effectively combines CLIP with a deeper exploitation of the Hierarchical class structure via Graph representation learning. We explore constructing the class hierarchy into a graph, with its nodes representing the textual or image features of each category. After passing through a graph encoder, the textual features incorporate hierarchical structure information, while the image features emphasize class-aware features derived from prototypes through the attention mechanism. Our approach demonstrates significant improvements on 11 diverse visual recognition benchmarks. Our codes are fully available at https://github.com/richard-peng-xia/HGCLIP. △ Less

Submitted 14 March, 2024; v1 submitted 23 November, 2023; originally announced November 2023.

A well-balanced lattice Boltzmann model for binary fluids based on the incompressible phase-field theory

Authors: Long Ju, Peiyao Liu, Bicheng Yan, Jin Bao, Shuyu Sun, Zhaoli Guo

Abstract: Spurious velocities arising from the imperfect offset of the undesired term at the discrete level are frequently observed in numerical simulations of equilibrium multiphase flow systems using the lattice Boltzmann equation (LBE) method. To capture the physical equilibrium state of two-phase fluid systems and eliminate spurious velocities, a well-balanced LBE model based on the incompressible phase… ▽ More Spurious velocities arising from the imperfect offset of the undesired term at the discrete level are frequently observed in numerical simulations of equilibrium multiphase flow systems using the lattice Boltzmann equation (LBE) method. To capture the physical equilibrium state of two-phase fluid systems and eliminate spurious velocities, a well-balanced LBE model based on the incompressible phase-field theory is developed. In this model, the equilibrium distribution function for the Cahn-Hilliard (CH) equation is designed by treating the convection term as a source to avoid the introduction of undesired terms, enabling achievement of possible discrete force balance. Furthermore, this approach allows for the attainment of a divergence-free velocity field, effectively mitigating the impact of artificial compression effects and enhancing numerical stability. Numerical tests, including a flat interface problem, a stationary droplet, and the coalescence of two droplets, demonstrate the well-balanced properties and improvements in the stability of the present model. △ Less

Submitted 17 November, 2023; originally announced November 2023.

A linear doubly stabilized Crank-Nicolson scheme for the Allen-Cahn equation with a general mobility

Authors: Dianming Hou, Zhonghua Qiao, Lili Ju

Abstract: In this paper, a linear second order numerical scheme is developed and investigated for the Allen-Cahn equation with a general positive mobility. In particular, our fully discrete scheme is mainly constructed based on the Crank-Nicolson formula for temporal discretization and the central finite difference method for spatial approximation, and two extra stabilizing terms are also introduced for the… ▽ More In this paper, a linear second order numerical scheme is developed and investigated for the Allen-Cahn equation with a general positive mobility. In particular, our fully discrete scheme is mainly constructed based on the Crank-Nicolson formula for temporal discretization and the central finite difference method for spatial approximation, and two extra stabilizing terms are also introduced for the purpose of improving numerical stability. The proposed scheme is shown to unconditionally preserve the maximum bound principle (MBP) under mild restrictions on the stabilization parameters, which is of practical importance for achieving good accuracy and stability simultaneously. With the help of uniform boundedness of the numerical solutions due to MBP, we then successfully derive $H^{1}$-norm and $L^{\infty}$-norm error estimates for the Allen-Cahn equation with a constant and a variable mobility, respectively. Moreover, the energy stability of the proposed scheme is also obtained in the sense that the discrete free energy is uniformly bounded by the one at the initial time plus a {\color{black}constant}. Finally, some numerical experiments are carried out to verify the theoretical results and illustrate the performance of the proposed scheme with a time adaptive strategy. △ Less

Submitted 30 October, 2023; originally announced October 2023.

Large Quantum Anomalous Hall Effect in Spin-Orbit Proximitized Rhombohedral Graphene

Authors: Tonghang Han, Zhengguang Lu, Yuxuan Yao, Jixiang Yang, Junseok Seo, Chiho Yoon, Kenji Watanabe, Takashi Taniguchi, Liang Fu, Fan Zhang, Long Ju

Abstract: The quantum anomalous Hall effect (QAHE) is a robust topological phenomenon featuring quantized Hall resistance at zero magnetic field. We report the QAHE in a rhombohedral pentalayer graphene/monolayer WS2 heterostructure. Distinct from other experimentally confirmed QAHE systems, this system has neither magnetic element nor moiré superlattice effect. The QAH states emerge at charge neutrality an… ▽ More The quantum anomalous Hall effect (QAHE) is a robust topological phenomenon featuring quantized Hall resistance at zero magnetic field. We report the QAHE in a rhombohedral pentalayer graphene/monolayer WS2 heterostructure. Distinct from other experimentally confirmed QAHE systems, this system has neither magnetic element nor moiré superlattice effect. The QAH states emerge at charge neutrality and feature Chern numbers C = +-5 at temperatures up to about 1.5 K. This large QAHE arises from the synergy of the electron correlation in intrinsic flat bands of pentalayer graphene, the gate-tuning effect, and the proximity-induced Ising spin-orbit-coupling. Our experiment demonstrates the potential of crystalline two-dimensional materials for intertwined electron correlation and band topology physics, and may enable a route for engineering chiral Majorana edge states. △ Less

Submitted 26 April, 2024; v1 submitted 26 October, 2023; originally announced October 2023.

Comments: to be published in Science

Journal ref: Science 384, 647-651 (2024)

NurViD: A Large Expert-Level Video Database for Nursing Procedure Activity Understanding

Authors: Ming Hu, Lin Wang, Siyuan Yan, Don Ma, Qingli Ren, Peng Xia, Wei Feng, Peibo Duan, Lie Ju, Zongyuan Ge

Abstract: The application of deep learning to nursing procedure activity understanding has the potential to greatly enhance the quality and safety of nurse-patient interactions. By utilizing the technique, we can facilitate training and education, improve quality control, and enable operational compliance monitoring. However, the development of automatic recognition systems in this field is currently hinder… ▽ More The application of deep learning to nursing procedure activity understanding has the potential to greatly enhance the quality and safety of nurse-patient interactions. By utilizing the technique, we can facilitate training and education, improve quality control, and enable operational compliance monitoring. However, the development of automatic recognition systems in this field is currently hindered by the scarcity of appropriately labeled datasets. The existing video datasets pose several limitations: 1) these datasets are small-scale in size to support comprehensive investigations of nursing activity; 2) they primarily focus on single procedures, lacking expert-level annotations for various nursing procedures and action steps; and 3) they lack temporally localized annotations, which prevents the effective localization of targeted actions within longer video sequences. To mitigate these limitations, we propose NurViD, a large video dataset with expert-level annotation for nursing procedure activity understanding. NurViD consists of over 1.5k videos totaling 144 hours, making it approximately four times longer than the existing largest nursing activity datasets. Notably, it encompasses 51 distinct nursing procedures and 177 action steps, providing a much more comprehensive coverage compared to existing datasets that primarily focus on limited procedures. To evaluate the efficacy of current deep learning methods on nursing activity understanding, we establish three benchmarks on NurViD: procedure recognition on untrimmed videos, procedure and action recognition on trimmed videos, and action detection. Our benchmark and code will be available at \url{https://github.com/minghu0830/NurViD-benchmark}. △ Less

Submitted 20 October, 2023; originally announced October 2023.

Comments: Accepted by NeurIPS 2023 Datasets and Benchmarks Track

Conservation law and Lie symmetry analysis of the (1+1) dimensional dispersive long-wave equation

Authors: Long Ju, Faiza Afzal, Yufeng Zhang

Abstract: In this paper, we mainly study the integrability of 1+1 dimensional dispersive long-wave equation. Firstly, the Lie symmetry analysis of the equation is carried out in the first part. And the optimal system of the equation is obtained according to the symmetry, and the invariant solution and the reduced form of the target equation are solved according to the results. Secondly, we use different met… ▽ More In this paper, we mainly study the integrability of 1+1 dimensional dispersive long-wave equation. Firstly, the Lie symmetry analysis of the equation is carried out in the first part. And the optimal system of the equation is obtained according to the symmetry, and the invariant solution and the reduced form of the target equation are solved according to the results. Secondly, we use different methods to solve the conservation law of the target equation. To begin with, we give the adjoint determination equation and adjoint symmetry of the 1+1 dimensional dispersive long-wave equation, and use the adjoint symmetry as the equation multiplier to find several conservation laws. Then we get a Lie bracket by using the relationship between the symmetry of the equation and the adjoint symmetry. Next its strict self-adjoint property is verified, and its conservation laws are solved by Ibragimov's method. Finally, the conservation laws of the target equation are solved by Noether's theorem. Thirdly we calculate some exact solutions of the target equation by three different methods. In the end of the paper, the Hamiltonian structure of the target equation, the generalized pre-symplectic that maps symmetries into adjoint-symmetries and some of its soliton solutions are calculated. In conclusion, we use the direct construction of conservation law method, Ibragimov's method and so on to solve some new conservation laws of 1+1 dimensional dispersive long-wave equation, use the relationship between symmetry and adjoint symmetry to construct the corresponding Lie brackets, and obtain some linear soliton solutions according to the conservation law of the equation. △ Less

Submitted 16 October, 2023; originally announced October 2023.

Rethinking Memory and Communication Cost for Efficient Large Language Model Training

Authors: Chan Wu, Hanxiao Zhang, Lin Ju, Jinjing Huang, Youshao Xiao, Zhaoxin Huan, Siyuan Li, Fanzhuang Meng, Lei Liang, Xiaolu Zhang, Jun Zhou

Abstract: Recently, various distributed strategies for large language model training have been proposed. However, these methods provided limited solutions for the trade-off between memory consumption and communication cost. In this paper, we rethink the impact of memory consumption and communication costs on the training speed of large language models, and propose a memory-communication balanced strategy se… ▽ More Recently, various distributed strategies for large language model training have been proposed. However, these methods provided limited solutions for the trade-off between memory consumption and communication cost. In this paper, we rethink the impact of memory consumption and communication costs on the training speed of large language models, and propose a memory-communication balanced strategy set Partial Redundancy Optimizer (PaRO). PaRO provides comprehensive options which reduces the amount and frequency of inter-group communication with minor memory redundancy by fine-grained sharding strategy, thereby improving the training efficiency in various training scenarios. Additionally, we propose a Hierarchical Overlapping Ring (HO-Ring) communication topology to enhance communication efficiency between nodes or across switches in large language model training. Our experiments demonstrate that PaRO significantly improves training throughput by 1.19x-2.50x compared to the SOTA method and achieves a near-linear scalability. The HO-Ring algorithm improves communication efficiency by 36.5% compared to the traditional Ring algorithm. △ Less

Submitted 30 October, 2023; v1 submitted 9 October, 2023; originally announced October 2023.

Deep surrogate model for learning Green's function associated with linear reaction-diffusion operator

Authors: Junqing Ji, Lili Ju, Xiaoping Zhang

Abstract: In this paper, we present a deep surrogate model for learning the Green's function associated with the reaction-diffusion operator in rectangular domain. The U-Net architecture is utilized to effectively capture the mapping from source to solution of the target partial differential equations (PDEs). To enable efficient training of the model without relying on labeled data, we propose a novel loss… ▽ More In this paper, we present a deep surrogate model for learning the Green's function associated with the reaction-diffusion operator in rectangular domain. The U-Net architecture is utilized to effectively capture the mapping from source to solution of the target partial differential equations (PDEs). To enable efficient training of the model without relying on labeled data, we propose a novel loss function that draws inspiration from traditional numerical methods used for solving PDEs. Furthermore, a hard encoding mechanism is employed to ensure that the predicted Green's function is perfectly matched with the boundary conditions. Based on the learned Green's function from the trained deep surrogate model, a fast solver is developed to solve the corresponding PDEs with different sources and boundary conditions. Various numerical examples are also provided to demonstrate the effectiveness of the proposed model. △ Less

Submitted 5 October, 2023; originally announced October 2023.

Comments: 18 pages, 15 figures

MSC Class: 65N80; 68T07

Energy-dissipative spectral renormalization exponential integrator method for gradient flow problems

Authors: Dianming Hou, Lili Ju, Zhonghua Qiao

Abstract: In this paper, we present a novel spectral renormalization exponential integrator method for solving gradient flow problems. Our method is specifically designed to simultaneously satisfy discrete analogues of the energy dissipation laws and achieve high-order accuracy in time. To accomplish this, our method first incorporates the energy dissipation law into the target gradient flow equation by int… ▽ More In this paper, we present a novel spectral renormalization exponential integrator method for solving gradient flow problems. Our method is specifically designed to simultaneously satisfy discrete analogues of the energy dissipation laws and achieve high-order accuracy in time. To accomplish this, our method first incorporates the energy dissipation law into the target gradient flow equation by introducing a time-dependent spectral renormalization (TDSR) factor. Then, the coupled equations are discretized using the spectral approximation in space and the exponential time differencing (ETD) in time. Finally, the resulting fully discrete nonlinear system is decoupled and solved using the Picard iteration at each time step. Furthermore, we introduce an extra enforcing term into the system for updating the TDSR factor, which greatly relaxes the time step size restriction of the proposed method and enhances its computational efficiency. Extensive numerical tests with various gradient flows are also presented to demonstrate the accuracy and effectiveness of our method as well as its high efficiency when combined with an adaptive time-stepping strategy for long-term simulations. △ Less

Submitted 1 October, 2023; originally announced October 2023.

Comments: 24 pages, 12 figures

Fractional Quantum Anomalous Hall Effect in a Graphene Moire Superlattice

Authors: Zhengguang Lu, Tonghang Han, Yuxuan Yao, Aidan P. Reddy, Jixiang Yang, Junseok Seo, Kenji Watanabe, Takashi Taniguchi, Liang Fu, Long Ju

Abstract: The fractional quantum anomalous Hall effect (FQAHE), the analog of the fractional quantum Hall effect1 at zero magnetic field, is predicted to exist in topological flat bands under spontaneous time-reversal-symmetry breaking. The demonstration of FQAHE could lead to non-Abelian anyons which form the basis of topological quantum computation. So far, FQAHE has been observed only in twisted MoTe2 (t… ▽ More The fractional quantum anomalous Hall effect (FQAHE), the analog of the fractional quantum Hall effect1 at zero magnetic field, is predicted to exist in topological flat bands under spontaneous time-reversal-symmetry breaking. The demonstration of FQAHE could lead to non-Abelian anyons which form the basis of topological quantum computation. So far, FQAHE has been observed only in twisted MoTe2 (t-MoTe2) at moire filling factor v > 1/2. Graphene-based moire superlattices are believed to host FQAHE with the potential advantage of superior material quality and higher electron mobility. Here we report the observation of integer and fractional QAH effects in a rhombohedral pentalayer graphene/hBN moire superlattice. At zero magnetic field, we observed plateaus of quantized Hall resistance Rxy = h/(ve^2) at filling factors v = 1, 2/3, 3/5, 4/7, 4/9, 3/7 and 2/5 of the moire superlattice respectively. These features are accompanied by clear dips in the longitudinal resistance Rxx. In addition, at zero magnetic field, Rxy equals 2h/e^2 at v = 1/2 and varies linearly with the filling factor-similar to the composite Fermi liquid (CFL) in the half-filled lowest Landau level at high magnetic fields. By tuning the gate displacement field D and v, we observed phase transitions from CFL and FQAH states to other correlated electron states. Our graphene system provides an ideal platform for exploring charge fractionalization and (non-Abelian) anyonic braiding at zero magnetic field, especially considering a lateral junction between FQAHE and superconducting regions in the same device. △ Less

Submitted 26 December, 2023; v1 submitted 29 September, 2023; originally announced September 2023.

Comments: Nature, in press

Towards Novel Class Discovery: A Study in Novel Skin Lesions Clustering

Authors: Wei Feng, Lie Ju, Lin Wang, Kaimin Song, Zongyuan Ge

Abstract: Existing deep learning models have achieved promising performance in recognizing skin diseases from dermoscopic images. However, these models can only recognize samples from predefined categories, when they are deployed in the clinic, data from new unknown categories are constantly emerging. Therefore, it is crucial to automatically discover and identify new semantic categories from new data. In t… ▽ More Existing deep learning models have achieved promising performance in recognizing skin diseases from dermoscopic images. However, these models can only recognize samples from predefined categories, when they are deployed in the clinic, data from new unknown categories are constantly emerging. Therefore, it is crucial to automatically discover and identify new semantic categories from new data. In this paper, we propose a new novel class discovery framework for automatically discovering new semantic classes from dermoscopy image datasets based on the knowledge of known classes. Specifically, we first use contrastive learning to learn a robust and unbiased feature representation based on all data from known and unknown categories. We then propose an uncertainty-aware multi-view cross pseudo-supervision strategy, which is trained jointly on all categories of data using pseudo labels generated by a self-labeling strategy. Finally, we further refine the pseudo label by aggregating neighborhood information through local sample similarity to improve the clustering performance of the model for unknown categories. We conducted extensive experiments on the dermatology dataset ISIC 2019, and the experimental results show that our approach can effectively leverage knowledge from known categories to discover new semantic categories. We also further validated the effectiveness of the different modules through extensive ablation experiments. Our code will be released soon. △ Less

Submitted 28 September, 2023; originally announced September 2023.

Comments: 10 pages, 1 figure,Accepted by miccai 2023

Charge redistribution, charge order and plasmon in La$_{2-x}$Sr$_{x}$CuO$_{4}$/La$_{2}$CuO$_{4}$ superlattices

Authors: Qizhi Li, Lele Ju, Hsiaoyu Huang, Yuxuan Zhang, Changwei Zou, Tianshuang Ren, A. Singh, Shilong Zhang, Qingzheng Qiu, Qian Xiao, Di-Jing Huang, Yanwu Xie, Zhen Chen, Yingying Peng

Abstract: Interfacial superconductors have the potential to revolutionize electronics, quantum computing, and fundamental physics due to their enhanced superconducting properties and ability to create new types of superconductors. The emergence of superconductivity at the interface of La$_{2-x}$Sr$_{x}$CuO$_{4}$/La$_{2}$CuO$_{4}$ (LSCO/LCO), with a T$_c$ enhancement of $\sim$ 10 K compared to the La… ▽ More Interfacial superconductors have the potential to revolutionize electronics, quantum computing, and fundamental physics due to their enhanced superconducting properties and ability to create new types of superconductors. The emergence of superconductivity at the interface of La$_{2-x}$Sr$_{x}$CuO$_{4}$/La$_{2}$CuO$_{4}$ (LSCO/LCO), with a T$_c$ enhancement of $\sim$ 10 K compared to the La$_{2-x}$Sr$_{x}$CuO$_{4}$ bulk single crystals, provides an exciting opportunity to study quantum phenomena in reduced dimensions. To investigate the carrier distribution and excitations in interfacial superconductors, we combine O K-edge resonant inelastic X-ray scattering and atomic-resolved scanning transmission electron microscopy measurements to study La$_{2-x}$Sr$_{x}$CuO$_{4}$/La$_{2}$CuO$_{4}$ superlattices (x=0.15, 0.45) and bulk La$_{1.55}$Sr$_{0.45}$CuO$_{4}$ films. We find direct evidence of charge redistribution, charge order and plasmon in LSCO/LCO superlattices. Notably, the observed behaviors of charge order and plasmon deviate from the anticipated properties of individual constituents or the average doping level of the superlattice. Instead, they conform harmoniously to the effective doping, a critical parameter governed by the T$_c$ of interfacial superconductors. △ Less

Submitted 4 September, 2023; originally announced September 2023.

Comments: 8 pages, 5 figures

Single and coupled cavity mode sensing schemes using a diagnostic field

Authors: Aaron W. Goodwin-Jones, Haochen Zhu, Carl Blair, Daniel D. Brown, Joris van Heijningen, Li Ju, Chunnong Zhao

Abstract: Precise optical mode matching is of critical importance in experiments using squeezed-vacuum states. Automatic spatial-mode matching schemes have the potential to reduce losses and improve loss stability. However, in quantum-enhanced coupled-cavity experiments, such as gravitational-wave detectors, one must also ensure that the sub-cavities are also mode matched. We propose a new mode sensing sche… ▽ More Precise optical mode matching is of critical importance in experiments using squeezed-vacuum states. Automatic spatial-mode matching schemes have the potential to reduce losses and improve loss stability. However, in quantum-enhanced coupled-cavity experiments, such as gravitational-wave detectors, one must also ensure that the sub-cavities are also mode matched. We propose a new mode sensing scheme, which works for simple and coupled cavities. The scheme requires no moving parts, nor tuning of Gouy phases. Instead a diagnostic field tuned to the HG20/LG10 mode frequency is used. The error signals are derived to be proportional to the difference in waist position, and difference in Rayleigh ranges, between the sub-cavity eigenmodes. The two error signals are separable by 90 degrees of demodulation phase. We demonstrate reasonable error signals for a simplified Einstein Telescope optical design. This work will facilitate routine use of extremely high levels of squeezing in current and future gravitational-wave detectors. △ Less

Submitted 29 August, 2023; originally announced August 2023.

Report number: LIGO-P2300010

A Joint Fermi-GBM and Swift-BAT Analysis of Gravitational-Wave Candidates from the Third Gravitational-wave Observing Run

Authors: C. Fletcher, J. Wood, R. Hamburg, P. Veres, C. M. Hui, E. Bissaldi, M. S. Briggs, E. Burns, W. H. Cleveland, M. M. Giles, A. Goldstein, B. A. Hristov, D. Kocevski, S. Lesage, B. Mailyan, C. Malacaria, S. Poolakkil, A. von Kienlin, C. A. Wilson-Hodge, The Fermi Gamma-ray Burst Monitor Team, M. Crnogorčević, J. DeLaunay, A. Tohuvavohu, R. Caputo, S. B. Cenko , et al. (1674 additional authors not shown)

Abstract: We present Fermi Gamma-ray Burst Monitor (Fermi-GBM) and Swift Burst Alert Telescope (Swift-BAT) searches for gamma-ray/X-ray counterparts to gravitational wave (GW) candidate events identified during the third observing run of the Advanced LIGO and Advanced Virgo detectors. Using Fermi-GBM on-board triggers and sub-threshold gamma-ray burst (GRB) candidates found in the Fermi-GBM ground analyses,… ▽ More We present Fermi Gamma-ray Burst Monitor (Fermi-GBM) and Swift Burst Alert Telescope (Swift-BAT) searches for gamma-ray/X-ray counterparts to gravitational wave (GW) candidate events identified during the third observing run of the Advanced LIGO and Advanced Virgo detectors. Using Fermi-GBM on-board triggers and sub-threshold gamma-ray burst (GRB) candidates found in the Fermi-GBM ground analyses, the Targeted Search and the Untargeted Search, we investigate whether there are any coincident GRBs associated with the GWs. We also search the Swift-BAT rate data around the GW times to determine whether a GRB counterpart is present. No counterparts are found. Using both the Fermi-GBM Targeted Search and the Swift-BAT search, we calculate flux upper limits and present joint upper limits on the gamma-ray luminosity of each GW. Given these limits, we constrain theoretical models for the emission of gamma-rays from binary black hole mergers. △ Less

Submitted 25 August, 2023; originally announced August 2023.

Orbital Multiferroicity in Pentalayer Rhombohedral Graphene

Authors: Tonghang Han, Zhengguang Lu, Giovanni Scuri, Jiho Sung, Jue Wang, Tianyi Han, Kenji Watanabe, Takashi Taniguchi, Liang Fu, Hongkun Park, Long Ju

Abstract: Ferroic orders describe spontaneous polarization of spin, charge, and lattice degrees of freedom in materials. Materials featuring multiple ferroic orders, known as multiferroics, play important roles in multi-functional electrical and magnetic device applications. 2D materials with honeycomb lattices offer exciting opportunities to engineer unconventional multiferroicity, where the ferroic orders… ▽ More Ferroic orders describe spontaneous polarization of spin, charge, and lattice degrees of freedom in materials. Materials featuring multiple ferroic orders, known as multiferroics, play important roles in multi-functional electrical and magnetic device applications. 2D materials with honeycomb lattices offer exciting opportunities to engineer unconventional multiferroicity, where the ferroic orders are driven purely by the orbital degrees of freedom but not electron spin. These include ferro-valleytricity corresponding to the electron valley and ferro-orbital-magnetism supported by quantum geometric effects. Such orbital multiferroics could offer strong valley-magnetic couplings and large responses to external fields-enabling device applications such as multiple-state memory elements, and electric control of valley and magnetic states. Here we report orbital multiferroicity in pentalayer rhombohedral graphene using low temperature magneto-transport measurements. We observed anomalous Hall signals Rxy with an exceptionally large Hall angle (tanΘH > 0.6) and orbital magnetic hysteresis at hole doping. There are four such states with different valley polarizations and orbital magnetizations, forming a valley-magnetic quartet. By sweeping the gate electric field E we observed a butterfly-shaped hysteresis of Rxy connecting the quartet. This hysteresis indicates a ferro-valleytronic order that couples to the composite field E\cdot B, but not the individual fields. Tuning E would switch each ferroic order independently, and achieve non-volatile switching of them together. Our observations demonstrate a new type of multiferroics and point to electrically tunable ultra-low power valleytronic and magnetic devices. △ Less

Submitted 30 September, 2023; v1 submitted 17 August, 2023; originally announced August 2023.

Journal ref: Nature 623, 41-47 (2023)

Search for Eccentric Black Hole Coalescences during the Third Observing Run of LIGO and Virgo

Authors: The LIGO Scientific Collaboration, the Virgo Collaboration, the KAGRA Collaboration, A. G. Abac, R. Abbott, H. Abe, F. Acernese, K. Ackley, C. Adamcewicz, S. Adhicary, N. Adhikari, R. X. Adhikari, V. K. Adkins, V. B. Adya, C. Affeldt, D. Agarwal, M. Agathos, O. D. Aguiar, I. Aguilar, L. Aiello, A. Ain, P. Ajith, T. Akutsu, S. Albanesi, R. A. Alfaidi , et al. (1750 additional authors not shown)

Abstract: Despite the growing number of confident binary black hole coalescences observed through gravitational waves so far, the astrophysical origin of these binaries remains uncertain. Orbital eccentricity is one of the clearest tracers of binary formation channels. Identifying binary eccentricity, however, remains challenging due to the limited availability of gravitational waveforms that include effect… ▽ More Despite the growing number of confident binary black hole coalescences observed through gravitational waves so far, the astrophysical origin of these binaries remains uncertain. Orbital eccentricity is one of the clearest tracers of binary formation channels. Identifying binary eccentricity, however, remains challenging due to the limited availability of gravitational waveforms that include effects of eccentricity. Here, we present observational results for a waveform-independent search sensitive to eccentric black hole coalescences, covering the third observing run (O3) of the LIGO and Virgo detectors. We identified no new high-significance candidates beyond those that were already identified with searches focusing on quasi-circular binaries. We determine the sensitivity of our search to high-mass (total mass $M>70$ $M_\odot$) binaries covering eccentricities up to 0.3 at 15 Hz orbital frequency, and use this to compare model predictions to search results. Assuming all detections are indeed quasi-circular, for our fiducial population model, we place an upper limit for the merger rate density of high-mass binaries with eccentricities $0 < e \leq 0.3$ at $0.33$ Gpc$^{-3}$ yr$^{-1}$ at 90\% confidence level. △ Less

Submitted 7 August, 2023; originally announced August 2023.

Comments: 24 pages, 5 figures

Report number: LIGO-P2300080

Long-time asymptotic behavior of the Hunter-Saxton equation

Authors: Luman Ju, Kai Xu, Engui Fan

Abstract: With $\bar{\partial}$-generalization of the Deift-Zhou steepest descent method, we investigate the long-time asymptotics of the solution to the Cauchy problem for the Hunter-Saxton (HS) equation \begin{eqnarray} &&u_{txx}-2ωu_x+2u_xu_{xx}+uu_{xxx}=0,\quad x\in \mathbb{R},\ t>0,\nonumber\\ &&u(x,0)=u_0(x), \nonumber \end{eqnarray} where $u_0\in H^{3,4}(\mathbb{R})$ and $ω>0$ is a constant. Using th… ▽ More With $\bar{\partial}$-generalization of the Deift-Zhou steepest descent method, we investigate the long-time asymptotics of the solution to the Cauchy problem for the Hunter-Saxton (HS) equation \begin{eqnarray} &&u_{txx}-2ωu_x+2u_xu_{xx}+uu_{xxx}=0,\quad x\in \mathbb{R},\ t>0,\nonumber\\ &&u(x,0)=u_0(x), \nonumber \end{eqnarray} where $u_0\in H^{3,4}(\mathbb{R})$ and $ω>0$ is a constant. Using the new scale $(y,t)$ and a series of deformations to a Riemann-Hilbert problem associated with the Cauchy problem, we obtain the long-time asymptotic approximations of the solution $u(x,t)$ in two space-time regions: The solution of the HS equation decays as the speed of $\mathcal{O}(t^{-1/2})$ in the region $y/t >0$; While in the region $y/t<0$, the solution of the HS equation is depicted by a parabolic cylinder model with an residual error order $\mathcal{O}(t^{-1+\frac{1}{2p}})$ with $ p>2$. △ Less

Submitted 14 December, 2023; v1 submitted 30 July, 2023; originally announced July 2023.

Comments: 32 pages

Developing and implementing an Einsteinian science curriculum from Years 3 to 10: Part B Teacher upskilling: response to training and teacher's classroom experience

Authors: Tejinder Kaur, Magdalena Kersting, Kyla Adams, David Blair, David Treagust, Anastasia Popkova, Shon Boublil, Jesse Santoso, Li Ju, Marjan Zadnik, David Wood, Elaine Horne, Darren McGoran, Susan Scott, Grady Venville

Abstract: Recent years have seen a growing interest in modernizing physics and science curricula around the world. While many science educators and curriculum developers design instructional resources to successfully introduce topics of Einsteinian physics to young learners, it is clear that successful curriculum development needs to rest on successful teacher professional development.Teachers with or witho… ▽ More Recent years have seen a growing interest in modernizing physics and science curricula around the world. While many science educators and curriculum developers design instructional resources to successfully introduce topics of Einsteinian physics to young learners, it is clear that successful curriculum development needs to rest on successful teacher professional development.Teachers with or without science backgrounds were trained in short professional learning workshops or completed micro-credential courses. The courses enabled teachers to gain knowledge and confidence to deliver the Einstein-First program. Detailed lesson plans and instructional videos for teachers define the lessons. Questionnaires were used to collect data, and teacher interviews were conducted following the various teacher training programs. The research results show that teachers effectively deliver the Einsteinian physics programs and that their subject matter and pedagogical content knowledge increased. In addition, teacher attitudes were favorable towards modernizing the physics curriculum. We conclude that it is feasible to upskill teachers from diverse backgrounds in Einsteinian physics and break the cycle that has inhibited the modernization of school curricula. △ Less

Submitted 19 December, 2023; v1 submitted 29 June, 2023; originally announced June 2023.

Developing and implementing an Einsteinian science curriculum from Years 3 to 10 : Part A Concepts, rationale and learning outcomes

Authors: Tejinder Kaur, Magdalena Kersting, David Blair, Kyla Adams, David Treagust, Jesse Santoso, Anastasia Popkova, Shon Boublil, Marjan Zadnik, Li Ju, David Wood, Elaine Horne, Darren McGoran

Abstract: There has been a growing realisation that school science curricula do not adequately reflect the revolutionary changes in our scientific understanding of the 20th century. This discrepancy between current school education and our modern scientific understanding has led to calls for the modernisation of the science curriculum. Although there have been attempts to introduce topics of Einsteinian phy… ▽ More There has been a growing realisation that school science curricula do not adequately reflect the revolutionary changes in our scientific understanding of the 20th century. This discrepancy between current school education and our modern scientific understanding has led to calls for the modernisation of the science curriculum. Although there have been attempts to introduce topics of Einsteinian physics (i.e., quantum physics and relativity) to school education, often at the secondary level, we still lack a seamless curriculum in which modern science concepts are gradually introduced in primary and middle schools. Guided by the Model of Educational Reconstruction and following a mixed-methods research design, the Einstein-First project aims to address this gap. Einstein-First has developed and implemented an Einsteinian curriculum from Years 3 to 10 (students aged 7- 16) that resolves the disconnect between science in schools and the modern world. This paper presents the concepts, rationale, and learning outcomes of the curriculum implementation in six Australian schools with 315 students across Years 3 to 10. Our findings lay the foundation for informed curriculum development towards a school education that can enhance students' understanding and appreciation of the fundamental concepts of modern science and its impact on our society. △ Less

Submitted 21 November, 2023; v1 submitted 29 June, 2023; originally announced June 2023.

LMPT: Prompt Tuning with Class-Specific Embedding Loss for Long-tailed Multi-Label Visual Recognition

Authors: Peng Xia, Di Xu, Ming Hu, Lie Ju, Zongyuan Ge

Abstract: Long-tailed multi-label visual recognition (LTML) task is a highly challenging task due to the label co-occurrence and imbalanced data distribution. In this work, we propose a unified framework for LTML, namely prompt tuning with class-specific embedding loss (LMPT), capturing the semantic feature interactions between categories by combining text and image modality data and improving the performan… ▽ More Long-tailed multi-label visual recognition (LTML) task is a highly challenging task due to the label co-occurrence and imbalanced data distribution. In this work, we propose a unified framework for LTML, namely prompt tuning with class-specific embedding loss (LMPT), capturing the semantic feature interactions between categories by combining text and image modality data and improving the performance synchronously on both head and tail classes. Specifically, LMPT introduces the embedding loss function with class-aware soft margin and re-weighting to learn class-specific contexts with the benefit of textual descriptions (captions), which could help establish semantic relationships between classes, especially between the head and tail classes. Furthermore, taking into account the class imbalance, the distribution-balanced loss is adopted as the classification loss function to further improve the performance on the tail classes without compromising head classes. Extensive experiments are conducted on VOC-LT and COCO-LT datasets, which demonstrates that our method significantly surpasses the previous state-of-the-art methods and zero-shot CLIP in LTML. Our codes are fully public at https://github.com/richard-peng-xia/LMPT. △ Less

Submitted 18 June, 2024; v1 submitted 8 May, 2023; originally announced May 2023.

Comments: Accepted by 3rd Workshop on Advances in Language and Vision Research (ALVR) @ ACL 2024

Correlated Insulator and Chern Insulators in Pentalayer Rhombohedral Stacked Graphene

Authors: Tonghang Han, Zhengguang Lu, Giovanni Scuri, Jiho Sung, Jue Wang, Tianyi Han, Kenji Watanabe, Takashi Taniguchi, Hongkun Park, Long Ju

Abstract: Rhombohedral stacked multilayer graphene is an ideal platform to search for correlated electron phenomena, due to its pair of flat bands touching at zero energy and further tunability by an electric field. Furthermore, its valley-dependent Berry phase at zero energy points to possible topological states when the pseudospin symmetry is broken by electron correlation. However, experimental explorati… ▽ More Rhombohedral stacked multilayer graphene is an ideal platform to search for correlated electron phenomena, due to its pair of flat bands touching at zero energy and further tunability by an electric field. Furthermore, its valley-dependent Berry phase at zero energy points to possible topological states when the pseudospin symmetry is broken by electron correlation. However, experimental explorations of these opportunities are very limited so far, due to a lack of devices with optimized layer numbers and configurations. Here we present electron transport measurements of hBN-encapsulated pentalayer graphene at down to 100 milli-Kelvin. We observed a correlated insulating state with >MOhm resistance at zero charge density and zero displacement field, where the tight-binding calculation predicts a metallic ground state. By increasing the displacement field, we observed a Chern insulator state with C = -5 and two other states with C = -3 at a low magnetic field of ~1 Tesla. At high displacement fields and charge densities, we observed isospin-polarized quarter- and half-metals. Therefore, rhombohedral-stacked pentalayer graphene is the first graphene system to exhibit two different types of Fermi-surface instabilities: driven by a pair of flat bands touching at zero energy, and by the Stoner mechanism in a single flat band. Our results demonstrate a new direction to explore intertwined electron correlation and topology phenomena in natural graphitic materials without the need of moiré superlattice engineering. △ Less

Submitted 30 September, 2023; v1 submitted 3 May, 2023; originally announced May 2023.

Journal ref: Nature Nanotechnology 19, 181-187 (2024)

Deep Graph Reprogramming

Authors: Yongcheng Jing, Chongbin Yuan, Li Ju, Yiding Yang, Xinchao Wang, Dacheng Tao

Abstract: In this paper, we explore a novel model reusing task tailored for graph neural networks (GNNs), termed as "deep graph reprogramming". We strive to reprogram a pre-trained GNN, without amending raw node features nor model parameters, to handle a bunch of cross-level downstream tasks in various domains. To this end, we propose an innovative Data Reprogramming paradigm alongside a Model Reprogramming… ▽ More In this paper, we explore a novel model reusing task tailored for graph neural networks (GNNs), termed as "deep graph reprogramming". We strive to reprogram a pre-trained GNN, without amending raw node features nor model parameters, to handle a bunch of cross-level downstream tasks in various domains. To this end, we propose an innovative Data Reprogramming paradigm alongside a Model Reprogramming paradigm. The former one aims to address the challenge of diversified graph feature dimensions for various tasks on the input side, while the latter alleviates the dilemma of fixed per-task-per-model behavior on the model side. For data reprogramming, we specifically devise an elaborated Meta-FeatPadding method to deal with heterogeneous input dimensions, and also develop a transductive Edge-Slimming as well as an inductive Meta-GraPadding approach for diverse homogenous samples. Meanwhile, for model reprogramming, we propose a novel task-adaptive Reprogrammable-Aggregator, to endow the frozen model with larger expressive capacities in handling cross-domain tasks. Experiments on fourteen datasets across node/graph classification/regression, 3D object recognition, and distributed action recognition, demonstrate that the proposed methods yield gratifying results, on par with those by re-training from scratch. △ Less

Submitted 27 April, 2023; originally announced April 2023.

Comments: CVPR 2023 Highlight

Towards the Universal Defense for Query-Based Audio Adversarial Attacks

Authors: Feng Guo, Zheng Sun, Yuxuan Chen, Lei Ju

Abstract: Recently, studies show that deep learning-based automatic speech recognition (ASR) systems are vulnerable to adversarial examples (AEs), which add a small amount of noise to the original audio examples. These AE attacks pose new challenges to deep learning security and have raised significant concerns about deploying ASR systems and devices. The existing defense methods are either limited in appli… ▽ More Recently, studies show that deep learning-based automatic speech recognition (ASR) systems are vulnerable to adversarial examples (AEs), which add a small amount of noise to the original audio examples. These AE attacks pose new challenges to deep learning security and have raised significant concerns about deploying ASR systems and devices. The existing defense methods are either limited in application or only defend on results, but not on process. In this work, we propose a novel method to infer the adversary intent and discover audio adversarial examples based on the AEs generation process. The insight of this method is based on the observation: many existing audio AE attacks utilize query-based methods, which means the adversary must send continuous and similar queries to target ASR models during the audio AE generation process. Inspired by this observation, We propose a memory mechanism by adopting audio fingerprint technology to analyze the similarity of the current query with a certain length of memory query. Thus, we can identify when a sequence of queries appears to be suspectable to generate audio AEs. Through extensive evaluation on four state-of-the-art audio AE attacks, we demonstrate that on average our defense identify the adversary intent with over 90% accuracy. With careful regard for robustness evaluations, we also analyze our proposed defense and its strength to withstand two adaptive attacks. Finally, our scheme is available out-of-the-box and directly compatible with any ensemble of ASR defense models to uncover audio AE attacks effectively without model retraining. △ Less

Submitted 20 April, 2023; originally announced April 2023.

Comments: Submitted to Cybersecurity journal