-
Exaptation: Academic mentees' career pathway to be independent and impactful
Authors:
Yanmeng Xing,
Ye Sun,
Tongxin Pan,
Xianglong Liang,
Giacomo Livan,
Yifang Ma
Abstract:
In science, mentees often follow their mentors' career paths, but exceptional mentees frequently break from this routine, sometimes even outperforming their mentors. However, the pathways to independence for these excellent mentees and their interactions with mentors remain unclear. We analyzed the careers of over 500,000 mentees in Chemistry, Neuroscience, and Physics over the past 60 years to ex…
▽ More
In science, mentees often follow their mentors' career paths, but exceptional mentees frequently break from this routine, sometimes even outperforming their mentors. However, the pathways to independence for these excellent mentees and their interactions with mentors remain unclear. We analyzed the careers of over 500,000 mentees in Chemistry, Neuroscience, and Physics over the past 60 years to examine the strategies mentees employ in selecting research topics relative to their mentors, how these strategies evolve, and their resulting impact. Utilizing co-citation network analysis and a topic-specific impact allocation algorithm, we mapped the topic territory for each mentor-mentee pair and quantified their academic impact accrued within the topic. Our findings reveal mentees tend to engage with their mentors' less-dominated topics and explore new topics at the same time, and through this exaptive process, they begin to progressively establish their own research territories. This trend is particularly pronounced among those who outperform their mentors. Moreover, we identified an inverted U-shaped curve between the extent of topic divergence and the mentees' long-term impact, suggesting a moderate divergence from the mentors' research focus optimizes the mentees' academic impact. Finally, along the path to independence, increased coauthorship with mentors impedes the mentees' impact, whereas extending their collaboration networks with the mentors' former collaborators proves beneficial. These findings fill a crucial gap in understanding how mentees' research topic selection strategies affect academic success and offer valuable guidance for early-career researchers on pursuing independent research paths.
△ Less
Submitted 29 August, 2024;
originally announced August 2024.
-
Study of the rare decay $J/ψ\to μ^+μ^-μ^+μ^-$
Authors:
LHCb collaboration,
R. Aaij,
A. S. W. Abdelmotteleb,
C. Abellan Beteta,
F. Abudinén,
T. Ackernley,
A. A. Adefisoye,
B. Adeva,
M. Adinolfi,
P. Adlarson,
C. Agapopoulou,
C. A. Aidala,
Z. Ajaltouni,
S. Akar,
K. Akiba,
P. Albicocco,
J. Albrecht,
F. Alessio,
M. Alexander,
Z. Aliouche,
P. Alvarez Cartelle,
R. Amalric,
S. Amato,
J. L. Amey,
Y. Amhis
, et al. (1096 additional authors not shown)
Abstract:
The rare electromagnetic $J/ψ\to μ^+μ^-μ^+μ^-$ decay is observed with a significance greatly exceeding the discovery threshold, using proton-proton collision data collected by the LHCb experiment during 2016-2018 at a center-of-mass energy of 13 TeV, corresponding to an integrated luminosity of $5.4\,\text{fb}^{-1}$. The rate of this decay is measured relative to that of the $J/ψ\to μ^+μ^-$ mode.…
▽ More
The rare electromagnetic $J/ψ\to μ^+μ^-μ^+μ^-$ decay is observed with a significance greatly exceeding the discovery threshold, using proton-proton collision data collected by the LHCb experiment during 2016-2018 at a center-of-mass energy of 13 TeV, corresponding to an integrated luminosity of $5.4\,\text{fb}^{-1}$. The rate of this decay is measured relative to that of the $J/ψ\to μ^+μ^-$ mode. Using the QED model for the four-muon decay in the efficiency estimation, its branching fraction is determined to be \begin{equation*}
{\mathcal{B}}(J/ψ\to μ^+μ^-μ^+μ^-) = (1.13\pm0.10\pm0.05\pm0.01)\times 10^{-6}, \end{equation*} where the uncertainties are statistical, systematic and due to the uncertainty on the branching fraction of the $J/ψ\to μ^+μ^-$ decay.
△ Less
Submitted 29 August, 2024;
originally announced August 2024.
-
Making Large Language Models Better Planners with Reasoning-Decision Alignment
Authors:
Zhijian Huang,
Tao Tang,
Shaoxiang Chen,
Sihao Lin,
Zequn Jie,
Lin Ma,
Guangrun Wang,
Xiaodan Liang
Abstract:
Data-driven approaches for autonomous driving (AD) have been widely adopted in the past decade but are confronted with dataset bias and uninterpretability. Inspired by the knowledge-driven nature of human driving, recent approaches explore the potential of large language models (LLMs) to improve understanding and decision-making in traffic scenarios. They find that the pretrain-finetune paradigm o…
▽ More
Data-driven approaches for autonomous driving (AD) have been widely adopted in the past decade but are confronted with dataset bias and uninterpretability. Inspired by the knowledge-driven nature of human driving, recent approaches explore the potential of large language models (LLMs) to improve understanding and decision-making in traffic scenarios. They find that the pretrain-finetune paradigm of LLMs on downstream data with the Chain-of-Thought (CoT) reasoning process can enhance explainability and scene understanding. However, such a popular strategy proves to suffer from the notorious problems of misalignment between the crafted CoTs against the consequent decision-making, which remains untouched by previous LLM-based AD methods. To address this problem, we motivate an end-to-end decision-making model based on multimodality-augmented LLM, which simultaneously executes CoT reasoning and carries out planning results. Furthermore, we propose a reasoning-decision alignment constraint between the paired CoTs and planning results, imposing the correspondence between reasoning and decision-making. Moreover, we redesign the CoTs to enable the model to comprehend complex scenarios and enhance decision-making performance. We dub our proposed large language planners with reasoning-decision alignment as RDA-Driver. Experimental evaluations on the nuScenes and DriveLM-nuScenes benchmarks demonstrate the effectiveness of our RDA-Driver in enhancing the performance of end-to-end AD systems. Specifically, our RDA-Driver achieves state-of-the-art planning performance on the nuScenes dataset with 0.80 L2 error and 0.32 collision rate, and also achieves leading results on challenging DriveLM-nuScenes benchmarks with 0.82 L2 error and 0.38 collision rate.
△ Less
Submitted 25 August, 2024;
originally announced August 2024.
-
EasyControl: Transfer ControlNet to Video Diffusion for Controllable Generation and Interpolation
Authors:
Cong Wang,
Jiaxi Gu,
Panwen Hu,
Haoyu Zhao,
Yuanfan Guo,
Jianhua Han,
Hang Xu,
Xiaodan Liang
Abstract:
Following the advancements in text-guided image generation technology exemplified by Stable Diffusion, video generation is gaining increased attention in the academic community. However, relying solely on text guidance for video generation has serious limitations, as videos contain much richer content than images, especially in terms of motion. This information can hardly be adequately described w…
▽ More
Following the advancements in text-guided image generation technology exemplified by Stable Diffusion, video generation is gaining increased attention in the academic community. However, relying solely on text guidance for video generation has serious limitations, as videos contain much richer content than images, especially in terms of motion. This information can hardly be adequately described with plain text. Fortunately, in computer vision, various visual representations can serve as additional control signals to guide generation. With the help of these signals, video generation can be controlled in finer detail, allowing for greater flexibility for different applications. Integrating various controls, however, is nontrivial. In this paper, we propose a universal framework called EasyControl. By propagating and injecting condition features through condition adapters, our method enables users to control video generation with a single condition map. With our framework, various conditions including raw pixels, depth, HED, etc., can be integrated into different Unet-based pre-trained video diffusion models at a low practical cost. We conduct comprehensive experiments on public datasets, and both quantitative and qualitative results indicate that our method outperforms state-of-the-art methods. EasyControl significantly improves various evaluation metrics across multiple validation datasets compared to previous works. Specifically, for the sketch-to-video generation task, EasyControl achieves an improvement of 152.0 on FVD and 19.9 on IS, respectively, in UCF101 compared with VideoComposer. For fidelity, our model demonstrates powerful image retention ability, resulting in high FVD and IS in UCF101 and MSR-VTT compared to other image-to-video models.
△ Less
Submitted 23 August, 2024;
originally announced August 2024.
-
SIMPNet: Spatial-Informed Motion Planning Network
Authors:
Davood Soleymanzadeh,
Xiao Liang,
Minghui Zheng
Abstract:
Current robotic manipulators require fast and efficient motion-planning algorithms to operate in cluttered environments. State-of-the-art sampling-based motion planners struggle to scale to high-dimensional configuration spaces and are inefficient in complex environments. This inefficiency arises because these planners utilize either uniform or hand-crafted sampling heuristics within the configura…
▽ More
Current robotic manipulators require fast and efficient motion-planning algorithms to operate in cluttered environments. State-of-the-art sampling-based motion planners struggle to scale to high-dimensional configuration spaces and are inefficient in complex environments. This inefficiency arises because these planners utilize either uniform or hand-crafted sampling heuristics within the configuration space. To address these challenges, we present the Spatial-informed Motion Planning Network (SIMPNet). SIMPNet consists of a stochastic graph neural network (GNN)-based sampling heuristic for informed sampling within the configuration space. The sampling heuristic of SIMPNet encodes the workspace embedding into the configuration space through a cross-attention mechanism. It encodes the manipulator's kinematic structure into a graph, which is used to generate informed samples within the framework of sampling-based motion planning algorithms. We have evaluated the performance of SIMPNet using a UR5e robotic manipulator operating within simple and complex workspaces, comparing it against baseline state-of-the-art motion planners. The evaluation results show the effectiveness and advantages of the proposed planner compared to the baseline planners.
△ Less
Submitted 23 August, 2024;
originally announced August 2024.
-
Controllable Text Generation for Large Language Models: A Survey
Authors:
Xun Liang,
Hanyu Wang,
Yezhaohui Wang,
Shichao Song,
Jiawei Yang,
Simin Niu,
Jie Hu,
Dan Liu,
Shunyu Yao,
Feiyu Xiong,
Zhiyu Li
Abstract:
In Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated high text generation quality. However, in real-world applications, LLMs must meet increasingly complex requirements. Beyond avoiding misleading or inappropriate content, LLMs are also expected to cater to specific user needs, such as imitating particular writing styles or generating text with poetic richness. Thes…
▽ More
In Natural Language Processing (NLP), Large Language Models (LLMs) have demonstrated high text generation quality. However, in real-world applications, LLMs must meet increasingly complex requirements. Beyond avoiding misleading or inappropriate content, LLMs are also expected to cater to specific user needs, such as imitating particular writing styles or generating text with poetic richness. These varied demands have driven the development of Controllable Text Generation (CTG) techniques, which ensure that outputs adhere to predefined control conditions--such as safety, sentiment, thematic consistency, and linguistic style--while maintaining high standards of helpfulness, fluency, and diversity.
This paper systematically reviews the latest advancements in CTG for LLMs, offering a comprehensive definition of its core concepts and clarifying the requirements for control conditions and text quality. We categorize CTG tasks into two primary types: content control and attribute control. The key methods are discussed, including model retraining, fine-tuning, reinforcement learning, prompt engineering, latent space manipulation, and decoding-time intervention. We analyze each method's characteristics, advantages, and limitations, providing nuanced insights for achieving generation control. Additionally, we review CTG evaluation methods, summarize its applications across domains, and address key challenges in current research, including reduced fluency and practicality. We also propose several appeals, such as placing greater emphasis on real-world applications in future research. This paper aims to offer valuable guidance to researchers and developers in the field. Our reference list and Chinese version are open-sourced at https://github.com/IAAR-Shanghai/CTGSurvey.
△ Less
Submitted 22 August, 2024;
originally announced August 2024.
-
GarmentAligner: Text-to-Garment Generation via Retrieval-augmented Multi-level Corrections
Authors:
Shiyue Zhang,
Zheng Chong,
Xujie Zhang,
Hanhui Li,
Yuhao Cheng,
Yiqiang Yan,
Xiaodan Liang
Abstract:
General text-to-image models bring revolutionary innovation to the fields of arts, design, and media. However, when applied to garment generation, even the state-of-the-art text-to-image models suffer from fine-grained semantic misalignment, particularly concerning the quantity, position, and interrelations of garment components. Addressing this, we propose GarmentAligner, a text-to-garment diffus…
▽ More
General text-to-image models bring revolutionary innovation to the fields of arts, design, and media. However, when applied to garment generation, even the state-of-the-art text-to-image models suffer from fine-grained semantic misalignment, particularly concerning the quantity, position, and interrelations of garment components. Addressing this, we propose GarmentAligner, a text-to-garment diffusion model trained with retrieval-augmented multi-level corrections. To achieve semantic alignment at the component level, we introduce an automatic component extraction pipeline to obtain spatial and quantitative information of garment components from corresponding images and captions. Subsequently, to exploit component relationships within the garment images, we construct retrieval subsets for each garment by retrieval augmentation based on component-level similarity ranking and conduct contrastive learning to enhance the model perception of components from positive and negative samples. To further enhance the alignment of components across semantic, spatial, and quantitative granularities, we propose the utilization of multi-level correction losses that leverage detailed component information. The experimental findings demonstrate that GarmentAligner achieves superior fidelity and fine-grained semantic alignment when compared to existing competitors.
△ Less
Submitted 23 August, 2024; v1 submitted 22 August, 2024;
originally announced August 2024.
-
HoSZp: An Efficient Homomorphic Error-bounded Lossy Compressor for Scientific Data
Authors:
Tripti Agarwal,
Sheng Di,
Jiajun Huang,
Yafan Huang,
Ganesh Gopalakrishnan,
Robert Underwood,
Kai Zhao,
Xin Liang,
Guanpeng Li,
Franck Cappello
Abstract:
Error-bounded lossy compression has been a critical technique to significantly reduce the sheer amounts of simulation datasets for high-performance computing (HPC) scientific applications while effectively controlling the data distortion based on user-specified error bound. In many real-world use cases, users must perform computational operations on the compressed data (a.k.a. homomorphic compress…
▽ More
Error-bounded lossy compression has been a critical technique to significantly reduce the sheer amounts of simulation datasets for high-performance computing (HPC) scientific applications while effectively controlling the data distortion based on user-specified error bound. In many real-world use cases, users must perform computational operations on the compressed data (a.k.a. homomorphic compression). However, none of the existing error-bounded lossy compressors support the homomorphism, inevitably resulting in undesired decompression costs. In this paper, we propose a novel homomorphic error-bounded lossy compressor (called HoSZp), which supports not only error-bounding features but efficient computations (including negation, addition, multiplication, mean, variance, etc.) on the compressed data without the complete decompression step, which is the first attempt to the best of our knowledge. We develop several optimization strategies to maximize the overall compression ratio and execution performance. We evaluate HoSZp compared to other state-of-the-art lossy compressors based on multiple real-world scientific application datasets.
△ Less
Submitted 21 August, 2024;
originally announced August 2024.
-
Magnetization Plateaus in the Two-dimensional S = 1/2 Heisenberg Model with a 3$\times$3 Checkerboard Structure
Authors:
Xuyang Liang,
Dao-Xin Yao
Abstract:
We investigate the S=1/2 antiferromagnetic Heisenberg model with a 3$\times$3 checkerboard lattice structure in a longitudinal magnetic field. By using the stochastic series expansion quantum Monte Carlo (SSE-QMC) method, we obtain the properties of the non-plateau XY phase, 1/9, 3/9, 5/9, 7/9 magnetization plateau phases, and fully polarized phase. Then, we determine the precise phase transition…
▽ More
We investigate the S=1/2 antiferromagnetic Heisenberg model with a 3$\times$3 checkerboard lattice structure in a longitudinal magnetic field. By using the stochastic series expansion quantum Monte Carlo (SSE-QMC) method, we obtain the properties of the non-plateau XY phase, 1/9, 3/9, 5/9, 7/9 magnetization plateau phases, and fully polarized phase. Then, we determine the precise phase transition critical points belonging to the 3D XY universality class through finite-size scaling. Moreover, we study the longitudinal and transverse dynamic spin structure factors of this model in different phases. For the non-plateau XY phase, the energy spectra present a gap between the low-energy gapless branch and the high-energy part under the competition of magnetic field and interaction. The gapless branch can be described by the spin wave theory in the canted antiferromagnetic phase of the effective "block spin" model. In the magnetization plateau phase, we identify that the excitation arises from localized disturbances within the sublattice, which are capable of spreading in momentum space. This study offers theoretical insights and interpretations for the characteristics of the ground state and the inelastic neutron scattering spectrum in two-dimensional quantum magnetic materials. Specifically, it focuses on materials with a checkerboard unit cell structure and odd-spin configurations under the influence of a longitudinal magnetic field.
△ Less
Submitted 21 August, 2024;
originally announced August 2024.
-
E-Bench: Subjective-Aligned Benchmark Suite for Text-Driven Video Editing Quality Assessment
Authors:
Shangkun Sun,
Xiaoyu Liang,
Songlin Fan,
Wenxu Gao,
Wei Gao
Abstract:
Text-driven video editing has recently experienced rapid development. Despite this, evaluating edited videos remains a considerable challenge. Current metrics tend to fail to align with human perceptions, and effective quantitative metrics for video editing are still notably absent. To address this, we introduce E-Bench, a benchmark suite tailored to the assessment of text-driven video editing. Th…
▽ More
Text-driven video editing has recently experienced rapid development. Despite this, evaluating edited videos remains a considerable challenge. Current metrics tend to fail to align with human perceptions, and effective quantitative metrics for video editing are still notably absent. To address this, we introduce E-Bench, a benchmark suite tailored to the assessment of text-driven video editing. This suite includes E-Bench DB, a video quality assessment (VQA) database for video editing. E-Bench DB encompasses a diverse set of source videos featuring various motions and subjects, along with multiple distinct editing prompts, editing results from 8 different models, and the corresponding Mean Opinion Scores (MOS) from 24 human annotators. Based on E-Bench DB, we further propose E-Bench QA, a quantitative human-aligned measurement for the text-driven video editing task. In addition to the aesthetic, distortion, and other visual quality indicators that traditional VQA methods emphasize, E-Bench QA focuses on the text-video alignment and the relevance modeling between source and edited videos. It proposes a new assessment network for video editing that attains superior performance in alignment with human preferences. To the best of our knowledge, E-Bench introduces the first quality assessment dataset for video editing and an effective subjective-aligned quantitative metric for this domain. All data and code will be publicly available at https://github.com/littlespray/E-Bench.
△ Less
Submitted 21 August, 2024;
originally announced August 2024.
-
All Robots in One: A New Standard and Unified Dataset for Versatile, General-Purpose Embodied Agents
Authors:
Zhiqiang Wang,
Hao Zheng,
Yunshuang Nie,
Wenjun Xu,
Qingwei Wang,
Hua Ye,
Zhe Li,
Kaidong Zhang,
Xuewen Cheng,
Wanxi Dong,
Chang Cai,
Liang Lin,
Feng Zheng,
Xiaodan Liang
Abstract:
Embodied AI is transforming how AI systems interact with the physical world, yet existing datasets are inadequate for developing versatile, general-purpose agents. These limitations include a lack of standardized formats, insufficient data diversity, and inadequate data volume. To address these issues, we introduce ARIO (All Robots In One), a new data standard that enhances existing datasets by of…
▽ More
Embodied AI is transforming how AI systems interact with the physical world, yet existing datasets are inadequate for developing versatile, general-purpose agents. These limitations include a lack of standardized formats, insufficient data diversity, and inadequate data volume. To address these issues, we introduce ARIO (All Robots In One), a new data standard that enhances existing datasets by offering a unified data format, comprehensive sensory modalities, and a combination of real-world and simulated data. ARIO aims to improve the training of embodied AI agents, increasing their robustness and adaptability across various tasks and environments. Building upon the proposed new standard, we present a large-scale unified ARIO dataset, comprising approximately 3 million episodes collected from 258 series and 321,064 tasks. The ARIO standard and dataset represent a significant step towards bridging the gaps of existing data resources. By providing a cohesive framework for data collection and representation, ARIO paves the way for the development of more powerful and versatile embodied AI agents, capable of navigating and interacting with the physical world in increasingly complex and diverse ways. The project is available on https://imaei.github.io/project_pages/ario/
△ Less
Submitted 20 August, 2024;
originally announced August 2024.
-
MUSE: Mamba is Efficient Multi-scale Learner for Text-video Retrieval
Authors:
Haoran Tang,
Meng Cao,
Jinfa Huang,
Ruyang Liu,
Peng Jin,
Ge Li,
Xiaodan Liang
Abstract:
Text-Video Retrieval (TVR) aims to align and associate relevant video content with corresponding natural language queries. Most existing TVR methods are based on large-scale pre-trained vision-language models (e.g., CLIP). However, due to the inherent plain structure of CLIP, few TVR methods explore the multi-scale representations which offer richer contextual information for a more thorough under…
▽ More
Text-Video Retrieval (TVR) aims to align and associate relevant video content with corresponding natural language queries. Most existing TVR methods are based on large-scale pre-trained vision-language models (e.g., CLIP). However, due to the inherent plain structure of CLIP, few TVR methods explore the multi-scale representations which offer richer contextual information for a more thorough understanding. To this end, we propose MUSE, a multi-scale mamba with linear computational complexity for efficient cross-resolution modeling. Specifically, the multi-scale representations are generated by applying a feature pyramid on the last single-scale feature map. Then, we employ the Mamba structure as an efficient multi-scale learner to jointly learn scale-wise representations. Furthermore, we conduct comprehensive studies to investigate different model structures and designs. Extensive results on three popular benchmarks have validated the superiority of MUSE.
△ Less
Submitted 20 August, 2024;
originally announced August 2024.
-
FancyVideo: Towards Dynamic and Consistent Video Generation via Cross-frame Textual Guidance
Authors:
Jiasong Feng,
Ao Ma,
Jing Wang,
Bo Cheng,
Xiaodan Liang,
Dawei Leng,
Yuhui Yin
Abstract:
Synthesizing motion-rich and temporally consistent videos remains a challenge in artificial intelligence, especially when dealing with extended durations. Existing text-to-video (T2V) models commonly employ spatial cross-attention for text control, equivalently guiding different frame generations without frame-specific textual guidance. Thus, the model's capacity to comprehend the temporal logic c…
▽ More
Synthesizing motion-rich and temporally consistent videos remains a challenge in artificial intelligence, especially when dealing with extended durations. Existing text-to-video (T2V) models commonly employ spatial cross-attention for text control, equivalently guiding different frame generations without frame-specific textual guidance. Thus, the model's capacity to comprehend the temporal logic conveyed in prompts and generate videos with coherent motion is restricted. To tackle this limitation, we introduce FancyVideo, an innovative video generator that improves the existing text-control mechanism with the well-designed Cross-frame Textual Guidance Module (CTGM). Specifically, CTGM incorporates the Temporal Information Injector (TII), Temporal Affinity Refiner (TAR), and Temporal Feature Booster (TFB) at the beginning, middle, and end of cross-attention, respectively, to achieve frame-specific textual guidance. Firstly, TII injects frame-specific information from latent features into text conditions, thereby obtaining cross-frame textual conditions. Then, TAR refines the correlation matrix between cross-frame textual conditions and latent features along the time dimension. Lastly, TFB boosts the temporal consistency of latent features. Extensive experiments comprising both quantitative and qualitative evaluations demonstrate the effectiveness of FancyVideo. Our video demo, code and model are available at https://360cvgroup.github.io/FancyVideo/.
△ Less
Submitted 16 August, 2024; v1 submitted 15 August, 2024;
originally announced August 2024.
-
WeKnow-RAG: An Adaptive Approach for Retrieval-Augmented Generation Integrating Web Search and Knowledge Graphs
Authors:
Weijian Xie,
Xuefeng Liang,
Yuhui Liu,
Kaihua Ni,
Hong Cheng,
Zetian Hu
Abstract:
Large Language Models (LLMs) have greatly contributed to the development of adaptive intelligent agents and are positioned as an important way to achieve Artificial General Intelligence (AGI). However, LLMs are prone to produce factually incorrect information and often produce "phantom" content that undermines their reliability, which poses a serious challenge for their deployment in real-world sc…
▽ More
Large Language Models (LLMs) have greatly contributed to the development of adaptive intelligent agents and are positioned as an important way to achieve Artificial General Intelligence (AGI). However, LLMs are prone to produce factually incorrect information and often produce "phantom" content that undermines their reliability, which poses a serious challenge for their deployment in real-world scenarios. Enhancing LLMs by combining external databases and information retrieval mechanisms is an effective path. To address the above challenges, we propose a new approach called WeKnow-RAG, which integrates Web search and Knowledge Graphs into a "Retrieval-Augmented Generation (RAG)" system. First, the accuracy and reliability of LLM responses are improved by combining the structured representation of Knowledge Graphs with the flexibility of dense vector retrieval. WeKnow-RAG then utilizes domain-specific knowledge graphs to satisfy a variety of queries and domains, thereby improving performance on factual information and complex reasoning tasks by employing multi-stage web page retrieval techniques using both sparse and dense retrieval methods. Our approach effectively balances the efficiency and accuracy of information retrieval, thus improving the overall retrieval process. Finally, we also integrate a self-assessment mechanism for the LLM to evaluate the trustworthiness of the answers it generates. Our approach proves its outstanding effectiveness in a wide range of offline experiments and online submissions.
△ Less
Submitted 27 August, 2024; v1 submitted 14 August, 2024;
originally announced August 2024.
-
A Novel Model of Charged Leptons
Authors:
Dianfu Wang,
Xiao Liang,
Yanqing Guo
Abstract:
A novel model of charged leptons is presented, which contains two basics hypotheses. The first hypothesis is that the Yukawa coupling between Higgs field and charged leptons is the weak interaction, the Higgs field is a scalar intermediate boson which changes the chirality of charged leptons in the weak interaction. The other hypothesis is that the flavor eigenstates of charged leptons are the sup…
▽ More
A novel model of charged leptons is presented, which contains two basics hypotheses. The first hypothesis is that the Yukawa coupling between Higgs field and charged leptons is the weak interaction, the Higgs field is a scalar intermediate boson which changes the chirality of charged leptons in the weak interaction. The other hypothesis is that the flavor eigenstates of charged leptons are the superposition states of left-handed and right-handed elementary Weyl spinors before the electroweak symmetry breaking. According to this model, the Yukawa coupling constants between Higgs field and three generations of charged leptons are considered to be a universal constant, and the difference of the masses of different charged leptons is due to the different left-right mixing angles of their flavor eigenstates.
△ Less
Submitted 15 August, 2024; v1 submitted 11 August, 2024;
originally announced August 2024.
-
Period-doubling in the phase dynamics of a shunted HgTe quantum well Josephson junction
Authors:
Wei Liu,
Stanislau U. Piatrusha,
Xianhu Liang,
Sandeep Upadhyay,
Lena Fürst,
Charles Gould,
Johannes Kleinlein,
Hartmut Buhmann,
Martin P. Stehno,
Laurens W. Molenkamp
Abstract:
The fractional AC Josephson effect is a discerning property of topological superconductivity in hybrid Josephson junctions. Recent experimental observations of missing odd Shapiro steps and half Josephson frequency emission in various materials have sparked significant debate regarding their potential origin in the effect. In this study, we present microwave emission measurements on a resistively…
▽ More
The fractional AC Josephson effect is a discerning property of topological superconductivity in hybrid Josephson junctions. Recent experimental observations of missing odd Shapiro steps and half Josephson frequency emission in various materials have sparked significant debate regarding their potential origin in the effect. In this study, we present microwave emission measurements on a resistively shunted Josephson junction based on a HgTe quantum well. We demonstrate that, with significant spurious inductance in the shunt wiring, the experiment operates in a nonlinear dynamic regime characterized by period-doubling. This leads to additional microwave emission peaks at half of the Josephson frequency, $f_J/2$, which can mimic the $4π$-periodicity of topological Andreev states. The observed current-voltage characteristics and emission spectra are well-described by a simple RCLSJ model. Furthermore, we show that the nonlinear dynamics of the junction can be controlled using gate voltage, magnetic field, and temperature, with our model accurately reproducing these effects without incorporating any topological attributes. Our observations urge caution in interpreting emission at $f_J/2$ as evidence for gapless Andreev bound states in topological junctions and suggest the appropriate parameter range for future experiments.
△ Less
Submitted 12 August, 2024;
originally announced August 2024.
-
Observation of muonic Dalitz decays of $χ_{b}$ mesons and precise spectroscopy of hidden-beauty states
Authors:
LHCb collaboration,
R. Aaij,
A. S. W. Abdelmotteleb,
C. Abellan Beteta,
F. Abudinén,
T. Ackernley,
A. A. Adefisoye,
B. Adeva,
M. Adinolfi,
P. Adlarson,
C. Agapopoulou,
C. A. Aidala,
Z. Ajaltouni,
S. Akar,
K. Akiba,
P. Albicocco,
J. Albrecht,
F. Alessio,
M. Alexander,
Z. Aliouche,
P. Alvarez Cartelle,
R. Amalric,
S. Amato,
J. L. Amey,
Y. Amhis
, et al. (1114 additional authors not shown)
Abstract:
The decays of the $χ_{b1}(1P)$, $χ_{b2}(1P)$, $χ_{b1}(2P)$ and $χ_{b2}(2P)$~mesons into the~$Υ(1S)μ^+μ^-$ final state are observed with a high significance using proton-proton collision data collected with the LHCb detector and corresponding to an integrated luminosity of 9fb$^{-1}$. The newly observed decays together with the $Υ(2S)\rightarrow Υ(1S)π^+π^-$ and $Υ(3S)\rightarrow Υ(2S)π^+π^-$ decay…
▽ More
The decays of the $χ_{b1}(1P)$, $χ_{b2}(1P)$, $χ_{b1}(2P)$ and $χ_{b2}(2P)$~mesons into the~$Υ(1S)μ^+μ^-$ final state are observed with a high significance using proton-proton collision data collected with the LHCb detector and corresponding to an integrated luminosity of 9fb$^{-1}$. The newly observed decays together with the $Υ(2S)\rightarrow Υ(1S)π^+π^-$ and $Υ(3S)\rightarrow Υ(2S)π^+π^-$ decay modes are used for precision measurements of the mass and mass splittings for the hidden-beauty states.
△ Less
Submitted 9 August, 2024;
originally announced August 2024.
-
The influence of advection on the propagation phenomena of reaction-diffusion equations with KPP-bistable nonlinearity
Authors:
Xing Liang,
Lei Zhang,
Mingmin Zhang
Abstract:
This paper is devoted to propagation phenomena for a reaction-diffusion-advection equation in a one-dimensional heterogeneous environment, where heterogeneity is reflected by the nonlinearity term -- being KPP type on $(-\infty, -L]$ and being bistable type on $[L,+\infty)$ for some $L>0$. A comprehensive analysis is presented on the influence of advection and heterogeneous reactions, based on var…
▽ More
This paper is devoted to propagation phenomena for a reaction-diffusion-advection equation in a one-dimensional heterogeneous environment, where heterogeneity is reflected by the nonlinearity term -- being KPP type on $(-\infty, -L]$ and being bistable type on $[L,+\infty)$ for some $L>0$. A comprehensive analysis is presented on the influence of advection and heterogeneous reactions, based on various values of the advection rate $c$. Denote by $c_m$ and $c_b$ the spreading speeds of KPP and bistable reactions, respectively. When $c>-c_m$, it is shown that propagation can always occur with leftward spreading speed $c_m+c$ and rightward spreading speed $\min\big(\max(c_b-c,0),c_m-c\big)$. Moreover, a logarithmic delay of the level sets in the left direction is discovered. When $c \le -c_m$, propagation phenomena are determined by the initial data and by the sign of $c_b$. In particular, when $c_b>0$, the leftward propagation speed is $c_b+c$ if the initial population is "large enough"; whereas extinction occurs if the initial value is located in the bistable region and is "relatively small". In addition, the attractiveness of the bistable traveling wave is obtained when the leftward spreading speed is $c_b+c$ and/or when the rightward spreading speed is $c_b-c$.
△ Less
Submitted 10 August, 2024; v1 submitted 6 August, 2024;
originally announced August 2024.
-
APTNESS: Incorporating Appraisal Theory and Emotion Support Strategies for Empathetic Response Generation
Authors:
Yuxuan Hu,
Minghuan Tan,
Chenwei Zhang,
Zixuan Li,
Xiaodan Liang,
Min Yang,
Chengming Li,
Xiping Hu
Abstract:
Empathetic response generation is designed to comprehend the emotions of others and select the most appropriate strategies to assist them in resolving emotional challenges. Empathy can be categorized into cognitive empathy and affective empathy. The former pertains to the ability to understand and discern the emotional issues and situations of others, while the latter involves the capacity to prov…
▽ More
Empathetic response generation is designed to comprehend the emotions of others and select the most appropriate strategies to assist them in resolving emotional challenges. Empathy can be categorized into cognitive empathy and affective empathy. The former pertains to the ability to understand and discern the emotional issues and situations of others, while the latter involves the capacity to provide comfort. To enhance one's empathetic abilities, it is essential to develop both these aspects. Therefore, we develop an innovative framework that combines retrieval augmentation and emotional support strategy integration. Our framework starts with the introduction of a comprehensive emotional palette for empathy. We then apply appraisal theory to decompose this palette and create a database of empathetic responses. This database serves as an external resource and enhances the LLM's empathy by integrating semantic retrieval mechanisms. Moreover, our framework places a strong emphasis on the proper articulation of response strategies. By incorporating emotional support strategies, we aim to enrich the model's capabilities in both cognitive and affective empathy, leading to a more nuanced and comprehensive empathetic response. Finally, we extract datasets ED and ET from the empathetic dialogue dataset \textsc{EmpatheticDialogues} and ExTES based on dialogue length. Experiments demonstrate that our framework can enhance the empathy ability of LLMs from both cognitive and affective empathy perspectives. Our code is released at https://github.com/CAS-SIAT-XinHai/APTNESS.
△ Less
Submitted 22 July, 2024;
originally announced July 2024.
-
Robust High-frequency Laser Phase Noise Suppression by Adaptive Pound-Drever-Hall Feedforward
Authors:
Yu-Xin Chao,
Zhen-Xing Hua,
Xin-Hui Liang,
Zong-Pei Yue,
Chen Jia,
Li You,
Meng Khoon Tey
Abstract:
Suppressing high-frequency laser phase noise, particularly at frequencies near and beyond typical feedback bandwidths of a few MHz, is a critical yet challenging task in many advanced applications. Feedforward-based methods generally outperform feedback in high-frequency range, but their performances are more susceptible to perturbations. In this work, we focus on the Pound-Drever-Hall (PDH)-feedf…
▽ More
Suppressing high-frequency laser phase noise, particularly at frequencies near and beyond typical feedback bandwidths of a few MHz, is a critical yet challenging task in many advanced applications. Feedforward-based methods generally outperform feedback in high-frequency range, but their performances are more susceptible to perturbations. In this work, we focus on the Pound-Drever-Hall (PDH)-feedforward method we demonstrated recently [Yu-Xin Chao et al., Optica 11(7), 945-950 (2024)] and analyze the factors that affect its long-term stability. By constructing a simple circuit allowing for adaptive control of the feedforward gain in response to power fluctuations of cavity transmission, we demonstrate a robust $\geq 40$~dB suppression of laser phase noise around 2~MHz and a noise suppression bandwidth up to 50~MHz. In comparison, when using normal PDH feedback, robust noise suppression of over 40 dB can only occur for frequencies below tens of kHz in most setups. Our findings may pave the way for general usage of PDH feedforward and allow for simple construction of low-noise lasers for precise quantum controls and precision metrology.
△ Less
Submitted 28 July, 2024;
originally announced July 2024.
-
Grasping Force Control and Adaptation for a Cable-Driven Robotic Hand
Authors:
Eric Mountain,
Ean Weise,
Sibo Tian,
Beiwen Li,
Xiao Liang,
Minghui Zheng
Abstract:
This paper introduces a unique force control and adaptation algorithm for a lightweight and low-complexity five-fingered robotic hand, namely an Integrated-Finger Robotic Hand (IFRH). The force control and adaptation algorithm is intuitive to design, easy to implement, and improves the grasping functionality through feedforward adaptation automatically. Specifically, we have extended Youla-paramet…
▽ More
This paper introduces a unique force control and adaptation algorithm for a lightweight and low-complexity five-fingered robotic hand, namely an Integrated-Finger Robotic Hand (IFRH). The force control and adaptation algorithm is intuitive to design, easy to implement, and improves the grasping functionality through feedforward adaptation automatically. Specifically, we have extended Youla-parameterization which is traditionally used in feedback controller design into a feedforward iterative learning control algorithm (ILC). The uniqueness of such an extension is that both the feedback and feedforward controllers are parameterized over one unified design parameter which can be easily customized based on the desired closed-loop performance. While Youla-parameterization and ILC have been explored in the past on various applications, our unique parameterization and computational methods make the design intuitive and easy to implement. This provides both robust and adaptive learning capabilities, and our application rivals the complexity of many robotic hand control systems. Extensive experimental tests have been conducted to validate the effectiveness of our method.
△ Less
Submitted 27 July, 2024;
originally announced July 2024.
-
Measurement of $D^0-\overline{D}^0$ mixing and search for $CP$ violation with $D^0\rightarrow K^+π^-$ decays
Authors:
LHCb collaboration,
R. Aaij,
A. S. W. Abdelmotteleb,
C. Abellan Beteta,
F. Abudinén,
T. Ackernley,
A. A. Adefisoye,
B. Adeva,
M. Adinolfi,
P. Adlarson,
C. Agapopoulou,
C. A. Aidala,
Z. Ajaltouni,
S. Akar,
K. Akiba,
P. Albicocco,
J. Albrecht,
F. Alessio,
M. Alexander,
Z. Aliouche,
P. Alvarez Cartelle,
R. Amalric,
S. Amato,
J. L. Amey,
Y. Amhis
, et al. (1065 additional authors not shown)
Abstract:
A measurement of the time-dependent ratio of the $D^0\rightarrow K^+π^-$ to $\overline{D}^0\rightarrow K^+π^-$ decay rates is reported. The analysis uses a sample of proton-proton collisions corresponding to an integrated luminosity of 6 fb$^-1$ recorded by the LHCb experiment from 2015 through 2018 at a center-of-mass energy of 13 TeV. The $D^0$ meson is required to originate from a…
▽ More
A measurement of the time-dependent ratio of the $D^0\rightarrow K^+π^-$ to $\overline{D}^0\rightarrow K^+π^-$ decay rates is reported. The analysis uses a sample of proton-proton collisions corresponding to an integrated luminosity of 6 fb$^-1$ recorded by the LHCb experiment from 2015 through 2018 at a center-of-mass energy of 13 TeV. The $D^0$ meson is required to originate from a $D^{*+}\rightarrow D^0π^+$ decay, such that its flavor at production is inferred from the charge of the accompanying pion. The measurement is performed simultaneously for the $K^+π^-$ and $K^-π^+$ final states, allowing both mixing and $CP$-violation parameters to be determined. The value of the ratio of the decay rates at production is determined to be $R_{Kπ} = (343.1 \pm 2.0) \times 10^{-5}$. The mixing parameters are measured to be $c_{Kπ} = (51.4 \pm 3.5) \times 10^{-4}$ and $c_{Kπ}^{\prime} = (13 \pm 4) \times 10^{-6}$, where $\sqrt{R_{Kπ}}c_{Kπ}$ is the linear coefficient of the expansion of the ratio as a function of decay time in units of the $D^0$ lifetime, and $c_{Kπ}^{\prime}$ is the quadratic coefficient, both averaged between the $K^+π^-$ and $K^-π^+$ final states. The precision is improved relative to the previous best measurement by approximately 60%. No evidence for $CP$ violation is found.
△ Less
Submitted 25 July, 2024;
originally announced July 2024.
-
Artificial Immunofluorescence in a Flash: Rapid Synthetic Imaging from Brightfield Through Residual Diffusion
Authors:
Xiaodan Xing,
Chunling Tang,
Siofra Murdoch,
Giorgos Papanastasiou,
Yunzhe Guo,
Xianglu Xiao,
Jan Cross-Zamirski,
Carola-Bibiane Schönlieb,
Kristina Xiao Liang,
Zhangming Niu,
Evandro Fei Fang,
Yinhai Wang,
Guang Yang
Abstract:
Immunofluorescent (IF) imaging is crucial for visualizing biomarker expressions, cell morphology and assessing the effects of drug treatments on sub-cellular components. IF imaging needs extra staining process and often requiring cell fixation, therefore it may also introduce artefects and alter endogenouous cell morphology. Some IF stains are expensive or not readily available hence hindering exp…
▽ More
Immunofluorescent (IF) imaging is crucial for visualizing biomarker expressions, cell morphology and assessing the effects of drug treatments on sub-cellular components. IF imaging needs extra staining process and often requiring cell fixation, therefore it may also introduce artefects and alter endogenouous cell morphology. Some IF stains are expensive or not readily available hence hindering experiments. Recent diffusion models, which synthesise high-fidelity IF images from easy-to-acquire brightfield (BF) images, offer a promising solution but are hindered by training instability and slow inference times due to the noise diffusion process. This paper presents a novel method for the conditional synthesis of IF images directly from BF images along with cell segmentation masks. Our approach employs a Residual Diffusion process that enhances stability and significantly reduces inference time. We performed a critical evaluation against other image-to-image synthesis models, including UNets, GANs, and advanced diffusion models. Our model demonstrates significant improvements in image quality (p<0.05 in MSE, PSNR, and SSIM), inference speed (26 times faster than competing diffusion models), and accurate segmentation results for both nuclei and cell bodies (0.77 and 0.63 mean IOU for nuclei and cell true positives, respectively). This paper is a substantial advancement in the field, providing robust and efficient tools for cell image analysis.
△ Less
Submitted 25 July, 2024;
originally announced July 2024.
-
DreamVTON: Customizing 3D Virtual Try-on with Personalized Diffusion Models
Authors:
Zhenyu Xie,
Haoye Dong,
Yufei Gao,
Zehua Ma,
Xiaodan Liang
Abstract:
Image-based 3D Virtual Try-ON (VTON) aims to sculpt the 3D human according to person and clothes images, which is data-efficient (i.e., getting rid of expensive 3D data) but challenging. Recent text-to-3D methods achieve remarkable improvement in high-fidelity 3D human generation, demonstrating its potential for 3D virtual try-on. Inspired by the impressive success of personalized diffusion models…
▽ More
Image-based 3D Virtual Try-ON (VTON) aims to sculpt the 3D human according to person and clothes images, which is data-efficient (i.e., getting rid of expensive 3D data) but challenging. Recent text-to-3D methods achieve remarkable improvement in high-fidelity 3D human generation, demonstrating its potential for 3D virtual try-on. Inspired by the impressive success of personalized diffusion models (e.g., Dreambooth and LoRA) for 2D VTON, it is straightforward to achieve 3D VTON by integrating the personalization technique into the diffusion-based text-to-3D framework. However, employing the personalized module in a pre-trained diffusion model (e.g., StableDiffusion (SD)) would degrade the model's capability for multi-view or multi-domain synthesis, which is detrimental to the geometry and texture optimization guided by Score Distillation Sampling (SDS) loss. In this work, we propose a novel customizing 3D human try-on model, named \textbf{DreamVTON}, to separately optimize the geometry and texture of the 3D human. Specifically, a personalized SD with multi-concept LoRA is proposed to provide the generative prior about the specific person and clothes, while a Densepose-guided ControlNet is exploited to guarantee consistent prior about body pose across various camera views. Besides, to avoid the inconsistent multi-view priors from the personalized SD dominating the optimization, DreamVTON introduces a template-based optimization mechanism, which employs mask templates for geometry shape learning and normal/RGB templates for geometry/texture details learning. Furthermore, for the geometry optimization phase, DreamVTON integrates a normal-style LoRA into personalized SD to enhance normal map generative prior, facilitating smooth geometry modeling.
△ Less
Submitted 23 July, 2024;
originally announced July 2024.
-
CatVTON: Concatenation Is All You Need for Virtual Try-On with Diffusion Models
Authors:
Zheng Chong,
Xiao Dong,
Haoxiang Li,
Shiyue Zhang,
Wenqing Zhang,
Xujie Zhang,
Hanqing Zhao,
Xiaodan Liang
Abstract:
Virtual try-on methods based on diffusion models achieve realistic try-on effects but often replicate the backbone network as a ReferenceNet or use additional image encoders to process condition inputs, leading to high training and inference costs. In this work, we rethink the necessity of ReferenceNet and image encoders and innovate the interaction between garment and person by proposing CatVTON,…
▽ More
Virtual try-on methods based on diffusion models achieve realistic try-on effects but often replicate the backbone network as a ReferenceNet or use additional image encoders to process condition inputs, leading to high training and inference costs. In this work, we rethink the necessity of ReferenceNet and image encoders and innovate the interaction between garment and person by proposing CatVTON, a simple and efficient virtual try-on diffusion model. CatVTON facilitates the seamless transfer of in-shop or worn garments of any category to target persons by simply concatenating them in spatial dimensions as inputs. The efficiency of our model is demonstrated in three aspects: (1) Lightweight network: Only the original diffusion modules are used, without additional network modules. The text encoder and cross-attentions for text injection in the backbone are removed, reducing the parameters by 167.02M. (2) Parameter-efficient training: We identified the try-on relevant modules through experiments and achieved high-quality try-on effects by training only 49.57M parameters, approximately 5.51 percent of the backbone network's parameters. (3) Simplified inference: CatVTON eliminates all unnecessary conditions and preprocessing steps, including pose estimation, human parsing, and text input, requiring only a garment reference, target person image, and mask for the virtual try-on process. Extensive experiments demonstrate that CatVTON achieves superior qualitative and quantitative results with fewer prerequisites and trainable parameters than baseline methods. Furthermore, CatVTON shows good generalization in in-the-wild scenarios despite using open-source datasets with only 73K samples.
△ Less
Submitted 21 July, 2024;
originally announced July 2024.
-
Large-vocabulary forensic pathological analyses via prototypical cross-modal contrastive learning
Authors:
Chen Shen,
Chunfeng Lian,
Wanqing Zhang,
Fan Wang,
Jianhua Zhang,
Shuanliang Fan,
Xin Wei,
Gongji Wang,
Kehan Li,
Hongshu Mu,
Hao Wu,
Xinggong Liang,
Jianhua Ma,
Zhenyuan Wang
Abstract:
Forensic pathology is critical in determining the cause and manner of death through post-mortem examinations, both macroscopic and microscopic. The field, however, grapples with issues such as outcome variability, laborious processes, and a scarcity of trained professionals. This paper presents SongCi, an innovative visual-language model (VLM) designed specifically for forensic pathology. SongCi u…
▽ More
Forensic pathology is critical in determining the cause and manner of death through post-mortem examinations, both macroscopic and microscopic. The field, however, grapples with issues such as outcome variability, laborious processes, and a scarcity of trained professionals. This paper presents SongCi, an innovative visual-language model (VLM) designed specifically for forensic pathology. SongCi utilizes advanced prototypical cross-modal self-supervised contrastive learning to enhance the accuracy, efficiency, and generalizability of forensic analyses. It was pre-trained and evaluated on a comprehensive multi-center dataset, which includes over 16 million high-resolution image patches, 2,228 vision-language pairs of post-mortem whole slide images (WSIs), and corresponding gross key findings, along with 471 distinct diagnostic outcomes. Our findings indicate that SongCi surpasses existing multi-modal AI models in many forensic pathology tasks, performs comparably to experienced forensic pathologists and significantly better than less experienced ones, and provides detailed multi-modal explainability, offering critical assistance in forensic investigations. To the best of our knowledge, SongCi is the first VLM specifically developed for forensic pathological analysis and the first large-vocabulary computational pathology (CPath) model that directly processes gigapixel WSIs in forensic science.
△ Less
Submitted 20 July, 2024;
originally announced July 2024.
-
Internal Consistency and Self-Feedback in Large Language Models: A Survey
Authors:
Xun Liang,
Shichao Song,
Zifan Zheng,
Hanyu Wang,
Qingchen Yu,
Xunkai Li,
Rong-Hua Li,
Peng Cheng,
Zhonghao Wang,
Feiyu Xiong,
Zhiyu Li
Abstract:
Large language models (LLMs) often exhibit deficient reasoning or generate hallucinations. To address these, studies prefixed with "Self-" such as Self-Consistency, Self-Improve, and Self-Refine have been initiated. They share a commonality: involving LLMs evaluating and updating themselves. Nonetheless, these efforts lack a unified perspective on summarization, as existing surveys predominantly f…
▽ More
Large language models (LLMs) often exhibit deficient reasoning or generate hallucinations. To address these, studies prefixed with "Self-" such as Self-Consistency, Self-Improve, and Self-Refine have been initiated. They share a commonality: involving LLMs evaluating and updating themselves. Nonetheless, these efforts lack a unified perspective on summarization, as existing surveys predominantly focus on categorization.
In this paper, we summarize a theoretical framework, Internal Consistency, offering explanations for reasoning deficiencies and hallucinations. Internal Consistency refers to the consistency in expressions among LLMs' latent, decoding, or response layers based on sampling methodologies. Then, we introduce another effective theoretical framework capable of mining Internal Consistency, named Self-Feedback. This framework consists of two modules: Self-Evaluation and Self-Update. The former captures Internal Consistency Signals, while the latter leverages the signals to enhance either the model's response or the model itself. This framework has been employed in numerous studies.
We systematically classify these studies by tasks and lines of work; summarize relevant evaluation methods and benchmarks; and delve into the concern, "Does Self-Feedback Really Work?" We also propose several critical viewpoints, including the "Hourglass Evolution of Internal Consistency", "Consistency Is (Almost) Correctness" hypothesis, and "The Paradox of Latent and Explicit Reasoning". The relevant resources are open-sourced at https://github.com/IAAR-Shanghai/ICSFSurvey.
△ Less
Submitted 29 August, 2024; v1 submitted 19 July, 2024;
originally announced July 2024.
-
Contrastive Learning with Counterfactual Explanations for Radiology Report Generation
Authors:
Mingjie Li,
Haokun Lin,
Liang Qiu,
Xiaodan Liang,
Ling Chen,
Abdulmotaleb Elsaddik,
Xiaojun Chang
Abstract:
Due to the common content of anatomy, radiology images with their corresponding reports exhibit high similarity. Such inherent data bias can predispose automatic report generation models to learn entangled and spurious representations resulting in misdiagnostic reports. To tackle these, we propose a novel \textbf{Co}unter\textbf{F}actual \textbf{E}xplanations-based framework (CoFE) for radiology r…
▽ More
Due to the common content of anatomy, radiology images with their corresponding reports exhibit high similarity. Such inherent data bias can predispose automatic report generation models to learn entangled and spurious representations resulting in misdiagnostic reports. To tackle these, we propose a novel \textbf{Co}unter\textbf{F}actual \textbf{E}xplanations-based framework (CoFE) for radiology report generation. Counterfactual explanations serve as a potent tool for understanding how decisions made by algorithms can be changed by asking ``what if'' scenarios. By leveraging this concept, CoFE can learn non-spurious visual representations by contrasting the representations between factual and counterfactual images. Specifically, we derive counterfactual images by swapping a patch between positive and negative samples until a predicted diagnosis shift occurs. Here, positive and negative samples are the most semantically similar but have different diagnosis labels. Additionally, CoFE employs a learnable prompt to efficiently fine-tune the pre-trained large language model, encapsulating both factual and counterfactual content to provide a more generalizable prompt representation. Extensive experiments on two benchmarks demonstrate that leveraging the counterfactual explanations enables CoFE to generate semantically coherent and factually complete reports and outperform in terms of language generation and clinical efficacy metrics.
△ Less
Submitted 19 July, 2024;
originally announced July 2024.
-
Observation of exotic $J/ψφ$ resonances in diffractive processes in proton-proton collisions
Authors:
LHCb collaboration,
R. Aaij,
A. S. W. Abdelmotteleb,
C. Abellan Beteta,
F. Abudinén,
T. Ackernley,
A. A. Adefisoye,
B. Adeva,
M. Adinolfi,
P. Adlarson,
C. Agapopoulou,
C. A. Aidala,
Z. Ajaltouni,
S. Akar,
K. Akiba,
P. Albicocco,
J. Albrecht,
F. Alessio,
M. Alexander,
Z. Aliouche,
P. Alvarez Cartelle,
R. Amalric,
S. Amato,
J. L. Amey,
Y. Amhis
, et al. (1068 additional authors not shown)
Abstract:
The first study of $J/ψφ$ production in diffractive processes in proton-proton collisions is presented. The study is based on an LHCb dataset recorded at centre-of-mass energy of 13 TeV, corresponding to an integrated luminosity of 5 fb$^{-1}$. The data disfavour a nonresonant $J/ψφ$ production but are consistent with a resonant model including several resonant states observed previously only in…
▽ More
The first study of $J/ψφ$ production in diffractive processes in proton-proton collisions is presented. The study is based on an LHCb dataset recorded at centre-of-mass energy of 13 TeV, corresponding to an integrated luminosity of 5 fb$^{-1}$. The data disfavour a nonresonant $J/ψφ$ production but are consistent with a resonant model including several resonant states observed previously only in $B^+ \to J/ψφK^+$ decays. The $χ_{c0}(4500)$ state is observed with a significance over $5σ$ and the $χ_{c1}(4274)$ is confirmed with a significance of more than $4σ$.
△ Less
Submitted 19 July, 2024;
originally announced July 2024.
-
Study of charmonium production via the decay to $p\bar{p}$ at $\sqrt{s} = 13 TeV$
Authors:
LHCb collaboration,
R. Aaij,
A. S. W. Abdelmotteleb,
C. Abellan Beteta,
F. Abudinén,
T. Ackernley,
A. A. Adefisoye,
B. Adeva,
M. Adinolfi,
P. Adlarson,
C. Agapopoulou,
C. A. Aidala,
Z. Ajaltouni,
S. Akar,
K. Akiba,
P. Albicocco,
J. Albrecht,
F. Alessio,
M. Alexander,
Z. Aliouche,
P. Alvarez Cartelle,
R. Amalric,
S. Amato,
J. L. Amey,
Y. Amhis
, et al. (1060 additional authors not shown)
Abstract:
Charmonium production cross-section in proton-proton collisions is measured at the centre-of-mass energy $\sqrt{s}=13\,TeV$ using decays to $p\bar{p}$ final state. The study is performed using a data sample corresponding to an integrated luminosity of $2.2\,{fb}^{-1}$ collected in 2018 with the $LHCb$ detector. The production cross-section of the $η_c$ meson is measured in a rapidity range of…
▽ More
Charmonium production cross-section in proton-proton collisions is measured at the centre-of-mass energy $\sqrt{s}=13\,TeV$ using decays to $p\bar{p}$ final state. The study is performed using a data sample corresponding to an integrated luminosity of $2.2\,{fb}^{-1}$ collected in 2018 with the $LHCb$ detector. The production cross-section of the $η_c$ meson is measured in a rapidity range of $2.0 < y < 4.0$ and in a transverse momentum range of $5.0 < p_{T} < 20.0\,{GeV/\it{c}}$, which is extended compared with previous $LHCb$ analyses. The differential cross-section is measured in bins of $p_{T}$ and, for the first time, of $y$. Upper limits, at 90% and 95% confidence levels, on the $η_c(2S)$ and $h_c(1P)$ prompt production cross-sections are determined for the first time.
△ Less
Submitted 19 July, 2024;
originally announced July 2024.
-
Observation of surface Fermi arcs in altermagnetic Weyl semimetal CrSb
Authors:
Wenlong Lu,
Shiyu Feng,
Yuzhi Wang,
Dong Chen,
Zihan Lin,
Xin Liang,
Siyuan Liu,
Wanxiang Feng,
Kohei Yamagami,
Junwei Liu,
Claudia Felser,
Quansheng Wu,
Junzhang Ma
Abstract:
As a special type of collinear antiferromagnetism (AFM), altermagnetism has garnered significant research interest recently. Altermagnets exhibit broken parity-time symmetry and zero net magnetization in real space, leading to substantial band splitting in momentum space even in the absence of spin-orbit coupling. Meanwhile, parity-time symmetry breaking always induce nontrivial band topology such…
▽ More
As a special type of collinear antiferromagnetism (AFM), altermagnetism has garnered significant research interest recently. Altermagnets exhibit broken parity-time symmetry and zero net magnetization in real space, leading to substantial band splitting in momentum space even in the absence of spin-orbit coupling. Meanwhile, parity-time symmetry breaking always induce nontrivial band topology such as Weyl nodes. While Weyl semimetal states and nodal lines have been theoretically proposed in altermagnets, rare reports of experimental observation have been made up to this point. Using ARPES and first-principles calculations, we systematically studied the electronic structure of the room-temperature altermagnet candidate CrSb. At generic locations in momentum space, we clearly observed band spin splitting. Furthermore, we identified discrete surface Fermi arcs on the (100) cleaved side surface close to the Fermi level originating from bulk band topology. Our results imply that CrSb contains interesting nontrivial topological Weyl physics, in addition to being an excellent room temperature altermagnet.
△ Less
Submitted 18 July, 2024;
originally announced July 2024.
-
Amplitude analysis of $B^+ \to ψ(2S) K^+ π^+ π^-$ decays
Authors:
LHCb collaboration,
R. Aaij,
A. S. W. Abdelmotteleb,
C. Abellan Beteta,
F. Abudinén,
T. Ackernley,
A. A. Adefisoye,
B. Adeva,
M. Adinolfi,
P. Adlarson,
C. Agapopoulou,
C. A. Aidala,
Z. Ajaltouni,
S. Akar,
K. Akiba,
P. Albicocco,
J. Albrecht,
F. Alessio,
M. Alexander,
Z. Aliouche,
P. Alvarez Cartelle,
R. Amalric,
S. Amato,
J. L. Amey,
Y. Amhis
, et al. (1092 additional authors not shown)
Abstract:
The first full amplitude analysis of $B^+ \to ψ(2S) K^+ π^+ π^-$ decays is performed using proton-proton collision data corresponding to an integrated luminosity of $9\,\text{fb}^{-1}$ recorded with the LHCb detector. The rich $K^+ π^+ π^-$ spectrum is studied and the branching fractions of the resonant substructure associated with the prominent $K_1(1270)^+$ contribution are measured. The data ca…
▽ More
The first full amplitude analysis of $B^+ \to ψ(2S) K^+ π^+ π^-$ decays is performed using proton-proton collision data corresponding to an integrated luminosity of $9\,\text{fb}^{-1}$ recorded with the LHCb detector. The rich $K^+ π^+ π^-$ spectrum is studied and the branching fractions of the resonant substructure associated with the prominent $K_1(1270)^+$ contribution are measured. The data cannot be described by conventional strange and charmonium resonances only. An amplitude model with 53 components is developed comprising 11 hidden-charm exotic hadrons. New production mechanisms for charged charmonium-like states are observed. Significant resonant activity with spin-parity $J^P = 1^+$ in the $ψ(2S) π^+$ system is confirmed and a multi-pole structure is demonstrated. The spectral decomposition of the $ψ(2S) π^+ π^-$ invariant-mass structure, dominated by $X^0 \to ψ(2S) ρ(770)^0$ decays, broadly resembles the $J/ψφ$ spectrum observed in $B^+ \to J/ψφK^+$ decays. Exotic $ψ(2S) K^+ π^-$ resonances are observed for the first time.
△ Less
Submitted 17 July, 2024;
originally announced July 2024.
-
Search for the rare $Λ_c^+ \to p μ^+ μ^-$ decay
Authors:
LHCb collaboration,
R. Aaij,
A. S. W. Abdelmotteleb,
C. Abellan Beteta,
F. Abudinén,
T. Ackernley,
A. A. Adefisoye,
B. Adeva,
M. Adinolfi,
P. Adlarson,
C. Agapopoulou,
C. A. Aidala,
Z. Ajaltouni,
S. Akar,
K. Akiba,
P. Albicocco,
J. Albrecht,
F. Alessio,
M. Alexander,
Z. Aliouche,
P. Alvarez Cartelle,
R. Amalric,
S. Amato,
J. L. Amey,
Y. Amhis
, et al. (1062 additional authors not shown)
Abstract:
A search for the nonresonant $Λ_c^+ \to p μ^+ μ^-$ decay is performed using proton-proton collision data recorded at a centre-of-mass energy of 13 TeV by the LHCb experiment, corresponding to an integrated luminosity of 5.4 fb$^{-1}$. No evidence for the decay is found in the dimuon invariant-mass regions where the expected contributions of resonances is subdominant. The upper limit on the branchi…
▽ More
A search for the nonresonant $Λ_c^+ \to p μ^+ μ^-$ decay is performed using proton-proton collision data recorded at a centre-of-mass energy of 13 TeV by the LHCb experiment, corresponding to an integrated luminosity of 5.4 fb$^{-1}$. No evidence for the decay is found in the dimuon invariant-mass regions where the expected contributions of resonances is subdominant. The upper limit on the branching fraction of the $Λ_c^+ \to p μ^+ μ^-$ decay is determined to be $2.9~(3.2) \times 10^{-8}$ at 90% (95%) confidence level. The branching fractions in the dimuon invariant-mass regions dominated by the $η$, $ρ$ and $ω$ resonances are also determined.
△ Less
Submitted 16 July, 2024;
originally announced July 2024.
-
Lessons from a human-in-the-loop machine learning approach for identifying vacant, abandoned, and deteriorated properties in Savannah, Georgia
Authors:
Xiaofan Liang,
Brian Brainerd,
Tara Hicks,
Clio Andris
Abstract:
Addressing strategies for managing vacant, abandoned, and deteriorated (VAD) properties is important for maintaining healthy communities. Yet, the process of identifying these properties can be difficult. Here, we create a human-in-the-loop machine learning (HITLML) model called VADecide and apply it to a parcel-level case study in Savannah, Georgia. The results show a higher prediction accuracy t…
▽ More
Addressing strategies for managing vacant, abandoned, and deteriorated (VAD) properties is important for maintaining healthy communities. Yet, the process of identifying these properties can be difficult. Here, we create a human-in-the-loop machine learning (HITLML) model called VADecide and apply it to a parcel-level case study in Savannah, Georgia. The results show a higher prediction accuracy than was achieved when using a machine learning model without human input in the training. The HITLML approach also reveals differences between machine vs. human-generated results. Our findings contribute to knowledge about the advantages and challenges of HITLML in urban planning.
[Accepted for Publication at a Peer Review Journal]
△ Less
Submitted 18 July, 2024; v1 submitted 15 July, 2024;
originally announced July 2024.
-
A Network Lens on Social Costs: Demolishing a Historic Street for a New Subway Station
Authors:
Xiaofan Liang,
Lu Chen,
Manying Lyu,
Yun Tian,
Changdong Ye
Abstract:
Urban redevelopment often involves a delicate balance between enhancing regional connectivity and preserving local social fabric. Through a case study in Guangzhou, China, we argue that demolishing a historic street to construct a new subway station shows competing interests between local government's priority to facilitate spatial connectivity and locals' priority to maintain a place for social i…
▽ More
Urban redevelopment often involves a delicate balance between enhancing regional connectivity and preserving local social fabric. Through a case study in Guangzhou, China, we argue that demolishing a historic street to construct a new subway station shows competing interests between local government's priority to facilitate spatial connectivity and locals' priority to maintain a place for social interaction and memories. We measure the social costs of the new subway station through a network lens, focusing on the loss of social ties and memories and low travel benefits of the new station for the local populations. We find that 1) the demolition will remove many small businesses that support locals' daily activities, social ties, and memories, and 2) the new station reduces travel distance and increases route options for passengers from other areas of the city more than locals nearby the demolition site. Our results contribute to a network-based framework and methodology to understand and contest inequality in expanding transportation network infrastructure in cities.
△ Less
Submitted 15 July, 2024;
originally announced July 2024.
-
Benchmarking LLMs for Optimization Modeling and Enhancing Reasoning via Reverse Socratic Synthesis
Authors:
Zhicheng Yang,
Yinya Huang,
Wei Shi,
Liang Feng,
Linqi Song,
Yiwei Wang,
Xiaodan Liang,
Jing Tang
Abstract:
Large language models (LLMs) have exhibited their problem-solving ability in mathematical reasoning. Solving realistic optimization (OPT) problems in industrial application scenarios requires advanced and applied math ability. However, current OPT benchmarks that merely solve linear programming are far from complex realistic situations. In this work, we propose E-OPT, a benchmark for end-to-end op…
▽ More
Large language models (LLMs) have exhibited their problem-solving ability in mathematical reasoning. Solving realistic optimization (OPT) problems in industrial application scenarios requires advanced and applied math ability. However, current OPT benchmarks that merely solve linear programming are far from complex realistic situations. In this work, we propose E-OPT, a benchmark for end-to-end optimization problem-solving with human-readable inputs and outputs. E-OPT contains rich optimization problems, including linear/nonlinear programming with/without table data, which can comprehensively evaluate LLMs' solving ability. In our benchmark, LLMs are required to correctly understand the problem in E-OPT and call code solver to get precise numerical answers. Furthermore, to alleviate the data scarcity for optimization problems, and to bridge the gap between open-source LLMs on a small scale (e.g., Llama-2-7b and Llama-3-8b) and closed-source LLMs (e.g., GPT-4), we further propose a novel data synthesis method namely ReSocratic. Unlike general data synthesis methods that proceed from questions to answers, ReSocratic first incrementally synthesizes optimization scenarios with mathematical formulations step by step and then back-translates the generated scenarios into questions. In such a way, we construct the ReSocratic-29k dataset from a small seed sample pool with the powerful open-source large model DeepSeek-V2. To demonstrate the effectiveness of ReSocratic, we conduct supervised fine-tuning with ReSocratic-29k on multiple open-source models. The results show that Llama3-8b is significantly improved from 13.6% to 51.7% on E-OPT, while DeepSeek-V2 reaches 61.0%, approaching 65.5% of GPT-4.
△ Less
Submitted 13 July, 2024;
originally announced July 2024.
-
HiRes-LLaVA: Restoring Fragmentation Input in High-Resolution Large Vision-Language Models
Authors:
Runhui Huang,
Xinpeng Ding,
Chunwei Wang,
Jianhua Han,
Yulong Liu,
Hengshuang Zhao,
Hang Xu,
Lu Hou,
Wei Zhang,
Xiaodan Liang
Abstract:
High-resolution inputs enable Large Vision-Language Models (LVLMs) to discern finer visual details, enhancing their comprehension capabilities. To reduce the training and computation costs caused by high-resolution input, one promising direction is to use sliding windows to slice the input into uniform patches, each matching the input size of the well-trained vision encoder. Although efficient, th…
▽ More
High-resolution inputs enable Large Vision-Language Models (LVLMs) to discern finer visual details, enhancing their comprehension capabilities. To reduce the training and computation costs caused by high-resolution input, one promising direction is to use sliding windows to slice the input into uniform patches, each matching the input size of the well-trained vision encoder. Although efficient, this slicing strategy leads to the fragmentation of original input, i.e., the continuity of contextual information and spatial geometry is lost across patches, adversely affecting performance in cross-patch context perception and position-specific tasks. To overcome these shortcomings, we introduce HiRes-LLaVA, a novel framework designed to efficiently process any size of high-resolution input without altering the original contextual and geometric information. HiRes-LLaVA comprises two innovative components: (i) a SliceRestore adapter that reconstructs sliced patches into their original form, efficiently extracting both global and local features via down-up-sampling and convolution layers, and (ii) a Self-Mining Sampler to compresses the vision tokens based on themselves, preserving the original context and positional information while reducing training overhead. To assess the ability of handling context fragmentation, we construct a new benchmark, EntityGrid-QA, consisting of edge-related and position-related tasks. Our comprehensive experiments demonstrate the superiority of HiRes-LLaVA on both existing public benchmarks and on EntityGrid-QA, particularly on document-oriented tasks, establishing new standards for handling high-resolution inputs.
△ Less
Submitted 11 July, 2024;
originally announced July 2024.
-
Establishing Rigorous and Cost-effective Clinical Trials for Artificial Intelligence Models
Authors:
Wanling Gao,
Yunyou Huang,
Dandan Cui,
Zhuoming Yu,
Wenjing Liu,
Xiaoshuang Liang,
Jiahui Zhao,
Jiyue Xie,
Hao Li,
Li Ma,
Ning Ye,
Yumiao Kang,
Dingfeng Luo,
Peng Pan,
Wei Huang,
Zhongmou Liu,
Jizhong Hu,
Gangyuan Zhao,
Chongrong Jiang,
Fan Huang,
Tianyi Wei,
Suqin Tang,
Bingjie Xia,
Zhifei Zhang,
Jianfeng Zhan
Abstract:
A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. Moreover, the crucial role of cl…
▽ More
A profound gap persists between artificial intelligence (AI) and clinical practice in medicine, primarily due to the lack of rigorous and cost-effective evaluation methodologies. State-of-the-art and state-of-the-practice AI model evaluations are limited to laboratory studies on medical datasets or direct clinical trials with no or solely patient-centered controls. Moreover, the crucial role of clinicians in collaborating with AI, pivotal for determining its impact on clinical practice, is often overlooked. For the first time, we emphasize the critical necessity for rigorous and cost-effective evaluation methodologies for AI models in clinical practice, featuring patient/clinician-centered (dual-centered) AI randomized controlled trials (DC-AI RCTs) and virtual clinician-based in-silico trials (VC-MedAI) as an effective proxy for DC-AI RCTs. Leveraging 7500 diagnosis records from two-step inaugural DC-AI RCTs across 14 medical centers with 125 clinicians, our results demonstrate the necessity of DC-AI RCTs and the effectiveness of VC-MedAI. Notably, VC-MedAI performs comparably to human clinicians, replicating insights and conclusions from prospective DC-AI RCTs. We envision DC-AI RCTs and VC-MedAI as pivotal advancements, presenting innovative and transformative evaluation methodologies for AI models in clinical practice, offering a preclinical-like setting mirroring conventional medicine, and reshaping development paradigms in a cost-effective and fast-iterative manner. Chinese Clinical Trial Registration: ChiCTR2400086816.
△ Less
Submitted 28 July, 2024; v1 submitted 11 July, 2024;
originally announced July 2024.
-
OV-DINO: Unified Open-Vocabulary Detection with Language-Aware Selective Fusion
Authors:
Hao Wang,
Pengzhen Ren,
Zequn Jie,
Xiao Dong,
Chengjian Feng,
Yinlong Qian,
Lin Ma,
Dongmei Jiang,
Yaowei Wang,
Xiangyuan Lan,
Xiaodan Liang
Abstract:
Open-vocabulary detection is a challenging task due to the requirement of detecting objects based on class names, including those not encountered during training. Existing methods have shown strong zero-shot detection capabilities through pre-training and pseudo-labeling on diverse large-scale datasets. However, these approaches encounter two main challenges: (i) how to effectively eliminate data…
▽ More
Open-vocabulary detection is a challenging task due to the requirement of detecting objects based on class names, including those not encountered during training. Existing methods have shown strong zero-shot detection capabilities through pre-training and pseudo-labeling on diverse large-scale datasets. However, these approaches encounter two main challenges: (i) how to effectively eliminate data noise from pseudo-labeling, and (ii) how to efficiently leverage the language-aware capability for region-level cross-modality fusion and alignment. To address these challenges, we propose a novel unified open-vocabulary detection method called OV-DINO, which is pre-trained on diverse large-scale datasets with language-aware selective fusion in a unified framework. Specifically, we introduce a Unified Data Integration (UniDI) pipeline to enable end-to-end training and eliminate noise from pseudo-label generation by unifying different data sources into detection-centric data format. In addition, we propose a Language-Aware Selective Fusion (LASF) module to enhance the cross-modality alignment through a language-aware query selection and fusion process. We evaluate the performance of the proposed OV-DINO on popular open-vocabulary detection benchmarks, achieving state-of-the-art results with an AP of 50.6% on the COCO benchmark and 40.1% on the LVIS benchmark in a zero-shot manner, demonstrating its strong generalization ability. Furthermore, the fine-tuned OV-DINO on COCO achieves 58.4% AP, outperforming many existing methods with the same backbone. The code for OV-DINO is available at https://github.com/wanghao9610/OV-DINO.
△ Less
Submitted 21 July, 2024; v1 submitted 10 July, 2024;
originally announced July 2024.
-
HumanRefiner: Benchmarking Abnormal Human Generation and Refining with Coarse-to-fine Pose-Reversible Guidance
Authors:
Guian Fang,
Wenbiao Yan,
Yuanfan Guo,
Jianhua Han,
Zutao Jiang,
Hang Xu,
Shengcai Liao,
Xiaodan Liang
Abstract:
Text-to-image diffusion models have significantly advanced in conditional image generation. However, these models usually struggle with accurately rendering images featuring humans, resulting in distorted limbs and other anomalies. This issue primarily stems from the insufficient recognition and evaluation of limb qualities in diffusion models. To address this issue, we introduce AbHuman, the firs…
▽ More
Text-to-image diffusion models have significantly advanced in conditional image generation. However, these models usually struggle with accurately rendering images featuring humans, resulting in distorted limbs and other anomalies. This issue primarily stems from the insufficient recognition and evaluation of limb qualities in diffusion models. To address this issue, we introduce AbHuman, the first large-scale synthesized human benchmark focusing on anatomical anomalies. This benchmark consists of 56K synthesized human images, each annotated with detailed, bounding-box level labels identifying 147K human anomalies in 18 different categories. Based on this, the recognition of human anomalies can be established, which in turn enhances image generation through traditional techniques such as negative prompting and guidance. To further boost the improvement, we propose HumanRefiner, a novel plug-and-play approach for the coarse-to-fine refinement of human anomalies in text-to-image generation. Specifically, HumanRefiner utilizes a self-diagnostic procedure to detect and correct issues related to both coarse-grained abnormal human poses and fine-grained anomaly levels, facilitating pose-reversible diffusion generation. Experimental results on the AbHuman benchmark demonstrate that HumanRefiner significantly reduces generative discrepancies, achieving a 2.9x improvement in limb quality compared to the state-of-the-art open-source generator SDXL and a 1.4x improvement over DALL-E 3 in human evaluations. Our data and code are available at https://github.com/Enderfga/HumanRefiner.
△ Less
Submitted 9 July, 2024;
originally announced July 2024.
-
Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AI
Authors:
Yang Liu,
Weixing Chen,
Yongjie Bai,
Xiaodan Liang,
Guanbin Li,
Wen Gao,
Liang Lin
Abstract:
Embodied Artificial Intelligence (Embodied AI) is crucial for achieving Artificial General Intelligence (AGI) and serves as a foundation for various applications that bridge cyberspace and the physical world. Recently, the emergence of Multi-modal Large Models (MLMs) and World Models (WMs) have attracted significant attention due to their remarkable perception, interaction, and reasoning capabilit…
▽ More
Embodied Artificial Intelligence (Embodied AI) is crucial for achieving Artificial General Intelligence (AGI) and serves as a foundation for various applications that bridge cyberspace and the physical world. Recently, the emergence of Multi-modal Large Models (MLMs) and World Models (WMs) have attracted significant attention due to their remarkable perception, interaction, and reasoning capabilities, making them a promising architecture for the brain of embodied agents. However, there is no comprehensive survey for Embodied AI in the era of MLMs. In this survey, we give a comprehensive exploration of the latest advancements in Embodied AI. Our analysis firstly navigates through the forefront of representative works of embodied robots and simulators, to fully understand the research focuses and their limitations. Then, we analyze four main research targets: 1) embodied perception, 2) embodied interaction, 3) embodied agent, and 4) sim-to-real adaptation, covering the state-of-the-art methods, essential paradigms, and comprehensive datasets. Additionally, we explore the complexities of MLMs in virtual and real embodied agents, highlighting their significance in facilitating interactions in dynamic digital and physical environments. Finally, we summarize the challenges and limitations of embodied AI and discuss their potential future directions. We hope this survey will serve as a foundational reference for the research community and inspire continued innovation. The associated project can be found at https://github.com/HCPLab-SYSU/Embodied_AI_Paper_List.
△ Less
Submitted 25 August, 2024; v1 submitted 9 July, 2024;
originally announced July 2024.
-
Affordances-Oriented Planning using Foundation Models for Continuous Vision-Language Navigation
Authors:
Jiaqi Chen,
Bingqian Lin,
Xinmin Liu,
Lin Ma,
Xiaodan Liang,
Kwan-Yee K. Wong
Abstract:
LLM-based agents have demonstrated impressive zero-shot performance in vision-language navigation (VLN) task. However, existing LLM-based methods often focus only on solving high-level task planning by selecting nodes in predefined navigation graphs for movements, overlooking low-level control in navigation scenarios. To bridge this gap, we propose AO-Planner, a novel Affordances-Oriented Planner…
▽ More
LLM-based agents have demonstrated impressive zero-shot performance in vision-language navigation (VLN) task. However, existing LLM-based methods often focus only on solving high-level task planning by selecting nodes in predefined navigation graphs for movements, overlooking low-level control in navigation scenarios. To bridge this gap, we propose AO-Planner, a novel Affordances-Oriented Planner for continuous VLN task. Our AO-Planner integrates various foundation models to achieve affordances-oriented low-level motion planning and high-level decision-making, both performed in a zero-shot setting. Specifically, we employ a Visual Affordances Prompting (VAP) approach, where the visible ground is segmented by SAM to provide navigational affordances, based on which the LLM selects potential candidate waypoints and plans low-level paths towards selected waypoints. We further propose a high-level PathAgent which marks planned paths into the image input and reasons the most probable path by comprehending all environmental information. Finally, we convert the selected path into 3D coordinates using camera intrinsic parameters and depth information, avoiding challenging 3D predictions for LLMs. Experiments on the challenging R2R-CE and RxR-CE datasets show that AO-Planner achieves state-of-the-art zero-shot performance (8.8% improvement on SPL). Our method can also serve as a data annotator to obtain pseudo-labels, distilling its waypoint prediction ability into a learning-based predictor. This new predictor does not require any waypoint data from the simulator and achieves 47% SR competing with supervised methods. We establish an effective connection between LLM and 3D world, presenting novel prospects for employing foundation models in low-level motion control.
△ Less
Submitted 20 August, 2024; v1 submitted 8 July, 2024;
originally announced July 2024.
-
FALIP: Visual Prompt as Foveal Attention Boosts CLIP Zero-Shot Performance
Authors:
Jiedong Zhuang,
Jiaqi Hu,
Lianrui Mu,
Rui Hu,
Xiaoyu Liang,
Jiangnan Ye,
Haoji Hu
Abstract:
CLIP has achieved impressive zero-shot performance after pre-training on a large-scale dataset consisting of paired image-text data. Previous works have utilized CLIP by incorporating manually designed visual prompts like colored circles and blur masks into the images to guide the model's attention, showing enhanced zero-shot performance in downstream tasks. Although these methods have achieved pr…
▽ More
CLIP has achieved impressive zero-shot performance after pre-training on a large-scale dataset consisting of paired image-text data. Previous works have utilized CLIP by incorporating manually designed visual prompts like colored circles and blur masks into the images to guide the model's attention, showing enhanced zero-shot performance in downstream tasks. Although these methods have achieved promising results, they inevitably alter the original information of the images, which can lead to failure in specific tasks. We propose a train-free method Foveal-Attention CLIP (FALIP), which adjusts the CLIP's attention by inserting foveal attention masks into the multi-head self-attention module. We demonstrate FALIP effectively boosts CLIP zero-shot performance in tasks such as referring expressions comprehension, image classification, and 3D point cloud recognition. Experimental results further show that FALIP outperforms existing methods on most metrics and can augment current methods to enhance their performance.
△ Less
Submitted 21 August, 2024; v1 submitted 7 July, 2024;
originally announced July 2024.
-
Pathfinder: Exploring Path Diversity for Assessing Internet Censorship Inconsistency
Authors:
Xiaoqin Liang,
Guannan Liu,
Lin Jin,
Shuai Hao,
Haining Wang
Abstract:
Internet censorship is typically enforced by authorities to achieve information control for a certain group of Internet users. So far existing censorship studies have primarily focused on country-level characterization because (1) in many cases, censorship is enabled by governments with nationwide policies and (2) it is usually hard to control how the probing packets are routed to trigger censorsh…
▽ More
Internet censorship is typically enforced by authorities to achieve information control for a certain group of Internet users. So far existing censorship studies have primarily focused on country-level characterization because (1) in many cases, censorship is enabled by governments with nationwide policies and (2) it is usually hard to control how the probing packets are routed to trigger censorship in different networks inside a country. However, the deployment and implementation of censorship could be highly diverse at the ISP level. In this paper, we investigate Internet censorship from a different perspective by scrutinizing the diverse censorship deployment inside a country. Specifically, by leveraging an end-to-end measurement framework, we deploy multiple geo-distributed back-end control servers to explore various paths from one single vantage point. The generated traffic with the same domain but different control servers' IPs could be forced to traverse different transit networks, thereby being examined by different censorship devices if present. Through our large-scale experiments and in-depth investigation, we reveal that the diversity of Internet censorship caused by different routing paths inside a country is prevalent, implying that (1) the implementations of centralized censorship are commonly incomplete or flawed and (2) decentralized censorship is also common. Moreover, we identify that different hosting platforms also result in inconsistent censorship activities due to different peering relationships with the ISPs in a country. Finally, we present extensive case studies in detail to illustrate the configurations that lead to censorship inconsistency and explore the causes.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
PianoBART: Symbolic Piano Music Generation and Understanding with Large-Scale Pre-Training
Authors:
Xiao Liang,
Zijian Zhao,
Weichao Zeng,
Yutong He,
Fupeng He,
Yiyi Wang,
Chengying Gao
Abstract:
Learning musical structures and composition patterns is necessary for both music generation and understanding, but current methods do not make uniform use of learned features to generate and comprehend music simultaneously. In this paper, we propose PianoBART, a pre-trained model that uses BART for both symbolic piano music generation and understanding. We devise a multi-level object selection str…
▽ More
Learning musical structures and composition patterns is necessary for both music generation and understanding, but current methods do not make uniform use of learned features to generate and comprehend music simultaneously. In this paper, we propose PianoBART, a pre-trained model that uses BART for both symbolic piano music generation and understanding. We devise a multi-level object selection strategy for different pre-training tasks of PianoBART, which can prevent information leakage or loss and enhance learning ability. The musical semantics captured in pre-training are fine-tuned for music generation and understanding tasks. Experiments demonstrate that PianoBART efficiently learns musical patterns and achieves outstanding performance in generating high-quality coherent pieces and comprehending music. Our code and supplementary material are available at https://github.com/RS2002/PianoBart.
△ Less
Submitted 25 June, 2024;
originally announced July 2024.
-
Picosecond lifetimes of hydrogen bonds in the halide perovskite CH$_3$NH$_3$PbBr$_3$
Authors:
Alejandro Garrote-Márquez,
Lucas Lodeiro,
Norge Cruz Hernández,
Xia Liang,
Aron Walsh,
Eduardo Menéndez-Proupin
Abstract:
The structures and properties of organic-inorganic perovskites are influenced by the hydrogen bonding between the organic cations and the inorganic octahedral networks. This study explores the dynamics of hydrogen bonds in CH$_3$NH$_3$PbBr$_3$ across a temperature range from 70 K to 350 K, using molecular dynamics simulations with machine-learning force fields. The results indicate that the lifeti…
▽ More
The structures and properties of organic-inorganic perovskites are influenced by the hydrogen bonding between the organic cations and the inorganic octahedral networks. This study explores the dynamics of hydrogen bonds in CH$_3$NH$_3$PbBr$_3$ across a temperature range from 70 K to 350 K, using molecular dynamics simulations with machine-learning force fields. The results indicate that the lifetime of hydrogen bonds decreases with increasing temperature from 7.6 ps (70 K) to 0.16 ps (350 K), exhibiting Arrhenius-type behaviour. The geometric conditions for hydrogen bonding, which include bond lengths and angles, maintain consistency across the full temperature range. The relevance of hydrogen bonds for the vibrational states of the material is also evidenced through a detailed analysis of the vibrational power spectra, demonstrating their significant effect on the physical properties for this class of perovskites.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
Non-Adversarial Learning: Vector-Quantized Common Latent Space for Multi-Sequence MRI
Authors:
Luyi Han,
Tao Tan,
Tianyu Zhang,
Xin Wang,
Yuan Gao,
Chunyao Lu,
Xinglong Liang,
Haoran Dou,
Yunzhi Huang,
Ritse Mann
Abstract:
Adversarial learning helps generative models translate MRI from source to target sequence when lacking paired samples. However, implementing MRI synthesis with adversarial learning in clinical settings is challenging due to training instability and mode collapse. To address this issue, we leverage intermediate sequences to estimate the common latent space among multi-sequence MRI, enabling the rec…
▽ More
Adversarial learning helps generative models translate MRI from source to target sequence when lacking paired samples. However, implementing MRI synthesis with adversarial learning in clinical settings is challenging due to training instability and mode collapse. To address this issue, we leverage intermediate sequences to estimate the common latent space among multi-sequence MRI, enabling the reconstruction of distinct sequences from the common latent space. We propose a generative model that compresses discrete representations of each sequence to estimate the Gaussian distribution of vector-quantized common (VQC) latent space between multiple sequences. Moreover, we improve the latent space consistency with contrastive learning and increase model stability by domain augmentation. Experiments using BraTS2021 dataset show that our non-adversarial model outperforms other GAN-based methods, and VQC latent space aids our model to achieve (1) anti-interference ability, which can eliminate the effects of noise, bias fields, and artifacts, and (2) solid semantic representation ability, with the potential of one-shot segmentation. Our code is publicly available.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
Instrumental Variable methods to target Hypothetical Estimands with longitudinal repeated measures data: Application to the STEP 1 trial
Authors:
Jack Bowden,
Jesper Madsen,
Bryan Goldman,
Aske Thorn Iversen,
Xiaoran Liang,
Stijn Vansteelandt
Abstract:
The STEP 1 randomized trial evaluated the effect of taking semaglutide vs placebo on body weight over a 68 week duration. As with any study evaluating an intervention delivered over a sustained period, non-adherence was observed. This was addressed in the original trial analysis within the Estimand Framework by viewing non-adherence as an intercurrent event. The primary analysis applied a treatmen…
▽ More
The STEP 1 randomized trial evaluated the effect of taking semaglutide vs placebo on body weight over a 68 week duration. As with any study evaluating an intervention delivered over a sustained period, non-adherence was observed. This was addressed in the original trial analysis within the Estimand Framework by viewing non-adherence as an intercurrent event. The primary analysis applied a treatment policy strategy which viewed it as an aspect of the treatment regimen, and thus made no adjustment for its presence. A supplementary analysis used a hypothetical strategy, targeting an estimand that would have been realised had all participants adhered, under the assumption that no post-baseline variables confounded adherence and change in body weight. In this paper we propose an alternative Instrumental Variable method to adjust for non-adherence which does not rely on the same `unconfoundedness' assumption and is less vulnerable to positivity violations (e.g., it can give valid results even under conditions where non-adherence is guaranteed). Unlike many previous Instrumental Variable approaches, it makes full use of the repeatedly measured outcome data, and allows for a time-varying effect of treatment adherence on a participant's weight. We show that it provides a natural vehicle for defining two distinct hypothetical estimands: the treatment effect if all participants would have adhered to semaglutide, and the treatment effect if all participants would have adhered to both semaglutide and placebo. When applied to the STEP 1 study, they both suggest a sustained, slowly decaying weight loss effect of semaglutide treatment.
△ Less
Submitted 3 July, 2024;
originally announced July 2024.
-
To Forget or Not? Towards Practical Knowledge Unlearning for Large Language Models
Authors:
Bozhong Tian,
Xiaozhuan Liang,
Siyuan Cheng,
Qingbin Liu,
Mengru Wang,
Dianbo Sui,
Xi Chen,
Huajun Chen,
Ningyu Zhang
Abstract:
Large Language Models (LLMs) trained on extensive corpora inevitably retain sensitive data, such as personal privacy information and copyrighted material. Recent advancements in knowledge unlearning involve updating LLM parameters to erase specific knowledge. However, current unlearning paradigms are mired in vague forgetting boundaries, often erasing knowledge indiscriminately. In this work, we i…
▽ More
Large Language Models (LLMs) trained on extensive corpora inevitably retain sensitive data, such as personal privacy information and copyrighted material. Recent advancements in knowledge unlearning involve updating LLM parameters to erase specific knowledge. However, current unlearning paradigms are mired in vague forgetting boundaries, often erasing knowledge indiscriminately. In this work, we introduce KnowUnDo, a benchmark containing copyrighted content and user privacy domains to evaluate if the unlearning process inadvertently erases essential knowledge. Our findings indicate that existing unlearning methods often suffer from excessive unlearning. To address this, we propose a simple yet effective method, MemFlex, which utilizes gradient information to precisely target and unlearn sensitive parameters. Experimental results show that MemFlex is superior to existing methods in both precise knowledge unlearning and general knowledge retaining of LLMs. Code and dataset will be released at https://github.com/zjunlp/KnowUnDo.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
Field Knowledge as a Dual to Distributed Knowledge: A Characterization by Weighted Modal Logic
Authors:
Xiaolong Liang,
Yì N. Wáng
Abstract:
The study of group knowledge concepts such as mutual, common, and distributed knowledge is well established within the discipline of epistemic logic. In this work, we incorporate epistemic abilities of agents to refine the formal definition of distributed knowledge and introduce a formal characterization of field knowledge. We propose that field knowledge serves as a dual to distributed knowledge.…
▽ More
The study of group knowledge concepts such as mutual, common, and distributed knowledge is well established within the discipline of epistemic logic. In this work, we incorporate epistemic abilities of agents to refine the formal definition of distributed knowledge and introduce a formal characterization of field knowledge. We propose that field knowledge serves as a dual to distributed knowledge. Our approach utilizes epistemic logics with various group knowledge constructs, interpreted through weighted models. We delve into the eight logics that stem from these considerations, explore their relative expressivity and develop sound and complete axiomatic systems.
△ Less
Submitted 30 June, 2024;
originally announced July 2024.
