Showing 1–50 of 88 results for author: Fu, P

Reasoning and Tools for Human-Level Forecasting

Authors: Elvis Hsieh, Preston Fu, Jonathan Chen

Abstract: Language models (LMs) trained on web-scale datasets are largely successful due to their ability to memorize large amounts of training data, even if only present in a few examples. These capabilities are often desirable in evaluation on tasks such as question answering but raise questions about whether these models can exhibit genuine reasoning or succeed only at mimicking patterns from the trainin… ▽ More Language models (LMs) trained on web-scale datasets are largely successful due to their ability to memorize large amounts of training data, even if only present in a few examples. These capabilities are often desirable in evaluation on tasks such as question answering but raise questions about whether these models can exhibit genuine reasoning or succeed only at mimicking patterns from the training data. This distinction is particularly salient in forecasting tasks, where the answer is not present in the training data, and the model must reason to make logical deductions. We present Reasoning and Tools for Forecasting (RTF), a framework of reasoning-and-acting (ReAct) agents that can dynamically retrieve updated information and run numerical simulation with equipped tools. We evaluate our model with questions from competitive forecasting platforms and demonstrate that our method is competitive with and can outperform human predictions. This suggests that LMs, with the right tools, can indeed think and adapt like humans, offering valuable insights for real-world decision-making. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Two-dimensional Weyl nodal-line semimetal and antihelical edge states in a modified Kane-Mele model

Authors: Xiaokang Dai, Pei-Hao Fu, Yee Sin Ang, Qinjun Chen

Abstract: The Kane-Mele model has been modified to achieve versatile topological phases. Previous work [Phys. Rev. Lett. 120, 156402 (2018)] introduced a staggered intrinsic spin-orbit coupling effect to generate pseudohelical edge states, with Rashba spin-orbit coupling facilitating spin flips in alternating sublattices. Our study demonstrates that, in the absence of Rashba spin-orbit coupling, the modifie… ▽ More The Kane-Mele model has been modified to achieve versatile topological phases. Previous work [Phys. Rev. Lett. 120, 156402 (2018)] introduced a staggered intrinsic spin-orbit coupling effect to generate pseudohelical edge states, with Rashba spin-orbit coupling facilitating spin flips in alternating sublattices. Our study demonstrates that, in the absence of Rashba spin-orbit coupling, the modified Kane-Mele model with staggered intrinsic spin-orbit coupling evolves into a $Z_{2}$ class topological metal, specifically a two-dimensional Weyl nodal-line semimetal. In a nanoribbon geometry, we predict the emergence of antihelical edge states, which support spin-polarized currents flowing in the same direction along parallel boundaries. Unlike pseudohelical edge states, antihelical edge states can be viewed as a superposition of two antichiral edge states related by time-reversal symmetry. However, the spin Hall conductance from antihelical edge states is not quantized due to the presence of gapless bulk states. Additionally, we examine the robustness of helical, pseudohelical, and antihelical edge states in the presence of nonmagnetic disorders, highlighting the particular fragility of antihelical edge states. Our findings enhance the understanding of the modified Kane-Mele model, providing new insights into its topological properties. △ Less

Submitted 11 August, 2024; v1 submitted 8 August, 2024; originally announced August 2024.

Comments: 10 pages, 8 figures

Floquet engineering of topological phase transitions in quantum spin Hall $α$-$T_{3}$ system

Authors: Kok Wai Lee, Mateo Jalen Andrew Calderon, Xiang-Long Yu, Ching Hua Lee, Yee Sin Ang, Pei-Hao Fu

Abstract: Floquet engineering of topological phase transitions driven by a high-frequency time-periodic field is a promising approach to realizing new topological phases of matter distinct from static states. Here, we theoretically investigate Floquet engineering topological phase transitions in the quantum spin Hall $α$-$T_{3}$ system driven by an off-resonant circularly polarized light. In addition to the… ▽ More Floquet engineering of topological phase transitions driven by a high-frequency time-periodic field is a promising approach to realizing new topological phases of matter distinct from static states. Here, we theoretically investigate Floquet engineering topological phase transitions in the quantum spin Hall $α$-$T_{3}$ system driven by an off-resonant circularly polarized light. In addition to the quantum spin (anomalous) Hall insulator phase with multiple helical (chiral) edge states, spin-polarized topological metallic phases are observed, where the bulk topological band gap of one spin sub-band overlaps with the other gapless spin sub-band. Moreover, with a staggered potential, the topological invariants of the system depend on whether the middle band is occupied because of the breaking of particle-hole symmetry. Our work highlights the significance of Floquet engineering in realizing new topological phases in $α$-$T_{3}$ lattices. △ Less

Submitted 27 August, 2024; v1 submitted 4 August, 2024; originally announced August 2024.

Comments: 11 pages, 6 figures

EndoFinder: Online Image Retrieval for Explainable Colorectal Polyp Diagnosis

Authors: Ruijie Yang, Yan Zhu, Peiyao Fu, Yizhe Zhang, Zhihua Wang, Quanlin Li, Pinghong Zhou, Xian Yang, Shuo Wang

Abstract: Determining the necessity of resecting malignant polyps during colonoscopy screen is crucial for patient outcomes, yet challenging due to the time-consuming and costly nature of histopathology examination. While deep learning-based classification models have shown promise in achieving optical biopsy with endoscopic images, they often suffer from a lack of explainability. To overcome this limitatio… ▽ More Determining the necessity of resecting malignant polyps during colonoscopy screen is crucial for patient outcomes, yet challenging due to the time-consuming and costly nature of histopathology examination. While deep learning-based classification models have shown promise in achieving optical biopsy with endoscopic images, they often suffer from a lack of explainability. To overcome this limitation, we introduce EndoFinder, a content-based image retrieval framework to find the 'digital twin' polyp in the reference database given a newly detected polyp. The clinical semantics of the new polyp can be inferred referring to the matched ones. EndoFinder pioneers a polyp-aware image encoder that is pre-trained on a large polyp dataset in a self-supervised way, merging masked image modeling with contrastive learning. This results in a generic embedding space ready for different downstream clinical tasks based on image retrieval. We validate the framework on polyp re-identification and optical biopsy tasks, with extensive experiments demonstrating that EndoFinder not only achieves explainable diagnostics but also matches the performance of supervised classification models. EndoFinder's reliance on image retrieval has the potential to support diverse downstream decision-making tasks during real-time colonoscopy procedures. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Comments: MICCAI 2024

Think out Loud: Emotion Deducing Explanation in Dialogues

Authors: Jiangnan Li, Zheng Lin, Lanrui Wang, Qingyi Si, Yanan Cao, Mo Yu, Peng Fu, Weiping Wang, Jie Zhou

Abstract: Humans convey emotions through daily dialogues, making emotion understanding a crucial step of affective intelligence. To understand emotions in dialogues, machines are asked to recognize the emotion for an utterance (Emotion Recognition in Dialogues, ERD); based on the emotion, then find causal utterances for the emotion (Emotion Cause Extraction in Dialogues, ECED). The setting of the two tasks… ▽ More Humans convey emotions through daily dialogues, making emotion understanding a crucial step of affective intelligence. To understand emotions in dialogues, machines are asked to recognize the emotion for an utterance (Emotion Recognition in Dialogues, ERD); based on the emotion, then find causal utterances for the emotion (Emotion Cause Extraction in Dialogues, ECED). The setting of the two tasks requires first ERD and then ECED, ignoring the mutual complement between emotion and cause. To fix this, some new tasks are proposed to extract them simultaneously. Although the current research on these tasks has excellent achievements, simply identifying emotion-related factors by classification modeling lacks realizing the specific thinking process of causes stimulating the emotion in an explainable way. This thinking process especially reflected in the reasoning ability of Large Language Models (LLMs) is under-explored. To this end, we propose a new task "Emotion Deducing Explanation in Dialogues" (EDEN). EDEN recognizes emotion and causes in an explicitly thinking way. That is, models need to generate an explanation text, which first summarizes the causes; analyzes the inner activities of the speakers triggered by the causes using common sense; then guesses the emotion accordingly. To support the study of EDEN, based on the existing resources in ECED, we construct two EDEN datasets by human effort. We further evaluate different models on EDEN and find that LLMs are more competent than conventional PLMs. Besides, EDEN can help LLMs achieve better recognition of emotions and causes, which explores a new research direction of explainable emotion understanding in dialogues. △ Less

Submitted 7 June, 2024; originally announced June 2024.

Light-PEFT: Lightening Parameter-Efficient Fine-Tuning via Early Pruning

Authors: Naibin Gu, Peng Fu, Xiyu Liu, Bowen Shen, Zheng Lin, Weiping Wang

Abstract: Parameter-efficient fine-tuning (PEFT) has emerged as the predominant technique for fine-tuning in the era of large language models. However, existing PEFT methods still have inadequate training efficiency. Firstly, the utilization of large-scale foundation models during the training process is excessively redundant for certain fine-tuning tasks. Secondly, as the model size increases, the growth i… ▽ More Parameter-efficient fine-tuning (PEFT) has emerged as the predominant technique for fine-tuning in the era of large language models. However, existing PEFT methods still have inadequate training efficiency. Firstly, the utilization of large-scale foundation models during the training process is excessively redundant for certain fine-tuning tasks. Secondly, as the model size increases, the growth in trainable parameters of empirically added PEFT modules becomes non-negligible and redundant, leading to inefficiency. To achieve task-specific efficient fine-tuning, we propose the Light-PEFT framework, which includes two methods: Masked Early Pruning of the Foundation Model and Multi-Granularity Early Pruning of PEFT. The Light-PEFT framework allows for the simultaneous estimation of redundant parameters in both the foundation model and PEFT modules during the early stage of training. These parameters can then be pruned for more efficient fine-tuning. We validate our approach on GLUE, SuperGLUE, QA tasks, and various models. With Light-PEFT, parameters of the foundation model can be pruned by up to over 40%, while still controlling trainable parameters to be only 25% of the original PEFT method. Compared to utilizing the PEFT method directly, Light-PEFT achieves training and inference speedup, reduces memory usage, and maintains comparable performance and the plug-and-play feature of PEFT. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Comments: Findings of ACL 2024

Towards Generalizable Multi-Object Tracking

Authors: Zheng Qin, Le Wang, Sanping Zhou, Panpan Fu, Gang Hua, Wei Tang

Abstract: Multi-Object Tracking MOT encompasses various tracking scenarios, each characterized by unique traits. Effective trackers should demonstrate a high degree of generalizability across diverse scenarios. However, existing trackers struggle to accommodate all aspects or necessitate hypothesis and experimentation to customize the association information motion and or appearance for a given scenario, le… ▽ More Multi-Object Tracking MOT encompasses various tracking scenarios, each characterized by unique traits. Effective trackers should demonstrate a high degree of generalizability across diverse scenarios. However, existing trackers struggle to accommodate all aspects or necessitate hypothesis and experimentation to customize the association information motion and or appearance for a given scenario, leading to narrowly tailored solutions with limited generalizability. In this paper, we investigate the factors that influence trackers generalization to different scenarios and concretize them into a set of tracking scenario attributes to guide the design of more generalizable trackers. Furthermore, we propose a point-wise to instance-wise relation framework for MOT, i.e., GeneralTrack, which can generalize across diverse scenarios while eliminating the need to balance motion and appearance. Thanks to its superior generalizability, our proposed GeneralTrack achieves state-of-the-art performance on multiple benchmarks and demonstrates the potential for domain generalization. https://github.com/qinzheng2000/GeneralTrack.git △ Less

Submitted 1 June, 2024; originally announced June 2024.

Comments: CVPR2024

BeACONS: A Blockchain-enabled Authentication and Communications Network for Scalable IoV

Authors: Qi Shi, Jingyi Sun, Hanwei Fu, Peizhe Fu, Jiayuan Ma, Hao Xu, Erwu Liu

Abstract: This paper introduces a novel blockchain-enabled authentication and communications network for scalable Internet of Vehicles, which aims to bolster security and confidentiality, diminish communications latency, and reduce dependence on centralised infrastructures like Certificate Authorities and Public Key Infrastructures by leveraging Blockchain-enabled Domain Name Services and Blockchain-enabled… ▽ More This paper introduces a novel blockchain-enabled authentication and communications network for scalable Internet of Vehicles, which aims to bolster security and confidentiality, diminish communications latency, and reduce dependence on centralised infrastructures like Certificate Authorities and Public Key Infrastructures by leveraging Blockchain-enabled Domain Name Services and Blockchain-enabled Mutual Authentication. The proposed network is structured into a primary layer, consisting of Road Side Units and edge servers as servers of Blockchain-enabled Domain Name Services for managing inter-vehicle communications identities, and a sub-layer within each vehicle for intra-vehicle communications via the Blockchain-enabled Mutual Authentication Protocol. This design facilitates secure connections across vehicles by coordinating between the layers, significantly improving communications security and efficiency. This study also evaluates Road Side Unit availability against the random distribution of Road Side Units along the route of different vehicles. The proposed model presents a novel pathway towards a decentralised, secure, and efficient Internet of Vehicles ecosystem, contributing to the advancement of autonomous and trustworthy vehicular networks. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Nonreciprocal quantum phase transition in a spinning microwave magnonic system

Authors: Ye-jun Xu, Long-hua Zhai, Peng Fu, Shou-jing Cheng, Guo-Qiang Zhang

Abstract: We propose how to achieve nonreciprocal quantum phase transition in a spinning microwave magnonic system composed of a spinning microwave resonator coupled with an yttrium iron garnet sphere with magnon Kerr effect. Sagnac-Fizeau shift caused by the spinning of the resonator brings about a significant modification in the critical driving strengths for second- and one-order quantum phase transition… ▽ More We propose how to achieve nonreciprocal quantum phase transition in a spinning microwave magnonic system composed of a spinning microwave resonator coupled with an yttrium iron garnet sphere with magnon Kerr effect. Sagnac-Fizeau shift caused by the spinning of the resonator brings about a significant modification in the critical driving strengths for second- and one-order quantum phase transitions, which means that the highly controllable quantum phase can be realized by the spinning speed of the resonator. More importantly, based on the difference in the detunings of the counterclockwise and clockwise modes induced by spinning direction of the resonator, the phase transition in this system is nonreciprocal, that is, the quantum phase transition occurs when the system is driven in one direction but not the other. Our work offers an alternative path to engineer and design nonreciprocal magnonic devices. △ Less

Submitted 14 May, 2024; originally announced May 2024.

Comments: 8 pages, 6 figures

Transverse Cooper-Pair Rectifier

Authors: Pei-Hao Fu, Jun-Feng Liu, Yong Xu, Ching Hua Lee, Yee Sin Ang

Abstract: Non-reciprocal devices are key components in modern electronics covering broad applications ranging from transistors to logic circuits thanks to the output rectified signal in the direction parallel to the input. In this work, we propose a transverse Cooper-pair rectifier in which a non-reciprocal current is perpendicular to the driving field, when inversion, time reversal, and mirror symmetries a… ▽ More Non-reciprocal devices are key components in modern electronics covering broad applications ranging from transistors to logic circuits thanks to the output rectified signal in the direction parallel to the input. In this work, we propose a transverse Cooper-pair rectifier in which a non-reciprocal current is perpendicular to the driving field, when inversion, time reversal, and mirror symmetries are broken simultaneously. The Blonder-Tinkham-Klapwijk formalism is developed to describe the transverse current-voltage relation in a normal-metal/superconductor tunneling junction, where symmetry constraints are achieved by an effective built-in supercurrent manifesting in an asymmetric and anisotropic Andreev reflection. The asymmetry in the Andreev reflection is induced when inversion and time reversal symmetry are broken by the supercurrent component parallel to the junction while the anisotropy occurs when the mirror symmetry with respect to the normal of the junction interface is broken by the perpendicular supercurrent component to the junction. Compared to the conventional longitudinal one, the transverse rectifier supports fully polarized diode efficiency and colossal nonreciprocal conductance rectification, completely decoupling the path of the input excitation from the output rectified signal. This work provides a formalism for realizing transverse non-reciprocity in superconducting junctions, which is expected to be achieved by modifying current experimental setups and may pave the way for future low-dissipation superconducting electronics. △ Less

Submitted 25 June, 2024; v1 submitted 7 May, 2024; originally announced May 2024.

Comments: 8 + 10 pages, 3 + 4 figures

A bound preserving cut discontinuous Galerkin method for one dimensional hyperbolic conservation laws

Authors: Pei Fu, Gunilla Kreiss, Sara Zahedi

Abstract: In this paper we present a family of high order cut finite element methods with bound preserving properties for hyperbolic conservation laws in one space dimension. The methods are based on the discontinuous Galerkin framework and use a regular background mesh, where interior boundaries are allowed to cut through the mesh arbitrarily. Our methods include ghost penalty stabilization to handle small… ▽ More In this paper we present a family of high order cut finite element methods with bound preserving properties for hyperbolic conservation laws in one space dimension. The methods are based on the discontinuous Galerkin framework and use a regular background mesh, where interior boundaries are allowed to cut through the mesh arbitrarily. Our methods include ghost penalty stabilization to handle small cut elements and a new reconstruction of the approximation on macro-elements, which are local patches consisting of cut and un-cut neighboring elements that are connected by stabilization. We show that the reconstructed solution retains conservation and order of convergence. Our lowest-order scheme results in a piecewise constant solution that satisfies a maximum principle for scalar hyperbolic conservation laws. When the lowest order scheme is applied to the Euler equations, the scheme is positivity preserving in the sense that positivity of pressure and density are retained. For the high-order schemes, suitable bound preserving limiters are applied to the reconstructed solution on macro-elements. In the scalar case, a maximum principle limiter is applied, which ensures that the limited approximation satisfies the maximum principle. Correspondingly, we use a positivity preserving limiter for the Euler equations and show that our scheme is positivity preserving. In the presence of shocks, additional limiting is needed to avoid oscillations, hence we apply a standard TVB limiter to the reconstructed solution. The time step restrictions are of the same order as for the corresponding discontinuous Galerkin methods on the background mesh. Numerical computations illustrate accuracy, bound preservation, and shock capturing capabilities of the proposed schemes. △ Less

Submitted 5 June, 2024; v1 submitted 22 April, 2024; originally announced April 2024.

Comments: 32

ODM: A Text-Image Further Alignment Pre-training Approach for Scene Text Detection and Spotting

Authors: Chen Duan, Pei Fu, Shan Guo, Qianyi Jiang, Xiaoming Wei

Abstract: In recent years, text-image joint pre-training techniques have shown promising results in various tasks. However, in Optical Character Recognition (OCR) tasks, aligning text instances with their corresponding text regions in images poses a challenge, as it requires effective alignment between text and OCR-Text (referring to the text in images as OCR-Text to distinguish from the text in natural lan… ▽ More In recent years, text-image joint pre-training techniques have shown promising results in various tasks. However, in Optical Character Recognition (OCR) tasks, aligning text instances with their corresponding text regions in images poses a challenge, as it requires effective alignment between text and OCR-Text (referring to the text in images as OCR-Text to distinguish from the text in natural language) rather than a holistic understanding of the overall image content. In this paper, we propose a new pre-training method called OCR-Text Destylization Modeling (ODM) that transfers diverse styles of text found in images to a uniform style based on the text prompt. With ODM, we achieve better alignment between text and OCR-Text and enable pre-trained models to adapt to the complex and diverse styles of scene text detection and spotting tasks. Additionally, we have designed a new labeling generation method specifically for ODM and combined it with our proposed Text-Controller module to address the challenge of annotation costs in OCR tasks, allowing a larger amount of unlabeled data to participate in pre-training. Extensive experiments on multiple public datasets demonstrate that our method significantly improves performance and outperforms current pre-training methods in scene text detection and spotting tasks. Code is available at https://github.com/PriNing/ODM. △ Less

Submitted 17 April, 2024; v1 submitted 1 March, 2024; originally announced March 2024.

Comments: Accepted by CVPR2024

Are Large Language Models Table-based Fact-Checkers?

Authors: Hangwen Zhang, Qingyi Si, Peng Fu, Zheng Lin, Weiping Wang

Abstract: Table-based Fact Verification (TFV) aims to extract the entailment relation between statements and structured tables. Existing TFV methods based on small-scaled models suffer from insufficient labeled data and weak zero-shot ability. Recently, the appearance of Large Language Models (LLMs) has gained lots of attraction in research fields. They have shown powerful zero-shot and in-context learning… ▽ More Table-based Fact Verification (TFV) aims to extract the entailment relation between statements and structured tables. Existing TFV methods based on small-scaled models suffer from insufficient labeled data and weak zero-shot ability. Recently, the appearance of Large Language Models (LLMs) has gained lots of attraction in research fields. They have shown powerful zero-shot and in-context learning abilities on several NLP tasks, but their potential on TFV is still unknown. In this work, we implement a preliminary study about whether LLMs are table-based fact-checkers. In detail, we design diverse prompts to explore how the in-context learning can help LLMs in TFV, i.e., zero-shot and few-shot TFV capability. Besides, we carefully design and construct TFV instructions to study the performance gain brought by the instruction tuning of LLMs. Experimental results demonstrate that LLMs can achieve acceptable results on zero-shot and few-shot TFV with prompt engineering, while instruction-tuning can stimulate the TFV capability significantly. We also make some valuable findings about the format of zero-shot prompts and the number of in-context examples. Finally, we analyze some possible directions to promote the accuracy of TFV via LLMs, which is beneficial to further research of table reasoning. △ Less

Submitted 4 February, 2024; originally announced February 2024.

Comments: CSCWD 2024

Object Attribute Matters in Visual Question Answering

Authors: Peize Li, Qingyi Si, Peng Fu, Zheng Lin, Yan Wang

Abstract: Visual question answering is a multimodal task that requires the joint comprehension of visual and textual information. However, integrating visual and textual semantics solely through attention layers is insufficient to comprehensively understand and align information from both modalities. Intuitively, object attributes can naturally serve as a bridge to unify them, which has been overlooked in p… ▽ More Visual question answering is a multimodal task that requires the joint comprehension of visual and textual information. However, integrating visual and textual semantics solely through attention layers is insufficient to comprehensively understand and align information from both modalities. Intuitively, object attributes can naturally serve as a bridge to unify them, which has been overlooked in previous research. In this paper, we propose a novel VQA approach from the perspective of utilizing object attribute, aiming to achieve better object-level visual-language alignment and multimodal scene understanding. Specifically, we design an attribute fusion module and a contrastive knowledge distillation module. The attribute fusion module constructs a multimodal graph neural network to fuse attributes and visual features through message passing. The enhanced object-level visual features contribute to solving fine-grained problem like counting-question. The better object-level visual-language alignment aids in understanding multimodal scenes, thereby improving the model's robustness. Furthermore, to augment scene understanding and the out-of-distribution performance, the contrastive knowledge distillation module introduces a series of implicit knowledge. We distill knowledge into attributes through contrastive loss, which further strengthens the representation learning of attribute features and facilitates visual-linguistic alignment. Intensive experiments on six datasets, COCO-QA, VQAv2, VQA-CPv2, VQA-CPv1, VQAvs and TDIUC, show the superiority of the proposed method. △ Less

Submitted 20 December, 2023; originally announced January 2024.

Comments: AAAI 2024

Interplay between Haldane and modified Haldane models in $α$-$T_{3}$ lattice: Band structures, phase diagrams and edge states

Authors: Kok Wai Lee, Pei-Hao Fu, Yee Sin Ang

Abstract: We study the topological properties of the Haldane and modified Haldane models in $α$-$T_{3}$ lattice. The band structures and phase diagrams of the system are investigated. Individually, each model undergoes a distinct phase transition: (i) the Haldane-only model experiences a topological phase transition from the Chern insulator ($\mathcal{C} = 1$) phase to the higher Chern insulator (… ▽ More We study the topological properties of the Haldane and modified Haldane models in $α$-$T_{3}$ lattice. The band structures and phase diagrams of the system are investigated. Individually, each model undergoes a distinct phase transition: (i) the Haldane-only model experiences a topological phase transition from the Chern insulator ($\mathcal{C} = 1$) phase to the higher Chern insulator ($\mathcal{C} = 2$) phase; while (ii) the modified-Haldane-only model experiences a phase transition from the topological metal ($\mathcal{C} = 2$) phase to the higher Chern insulator ($\mathcal{C} = 2$) phase and we show that $\mathcal{C}$ is insufficient to characterize this system because $\mathcal{C}$ remains unchanged before and after the phase transition. By plotting the Chern number and $\mathcal{C}$ phase diagram, we show that in the presence of both Haldane and modified Haldane models in the $α$-$T_{3}$ lattice, the interplay between the two models manifests three distinct topological phases, namely the $\mathcal{C} = 1$ Chern insulator (CI) phase, $\mathcal{C} = 2$ higher Chern insulator (HCI) phase and $\mathcal{C} = 2$ topological metal (TM) phase. These results are further supported by the $α$-$T_{3}$ zigzag edge states calculations. Our work elucidates the rich phase evolution of Haldane and modified Haldane models as $α$ varies continuously from $0$ to $1$ in an $α$-$T_3$ model. △ Less

Submitted 11 December, 2023; v1 submitted 1 December, 2023; originally announced December 2023.

Comments: 9 pages, 5 figures

A Spatio-Temporal Graph Convolutional Network for Gesture Recognition from High-Density Electromyography

Authors: Wenjuan Zhong, Yuyang Zhang, Peiwen Fu, Wenxuan Xiong, Mingming Zhang

Abstract: Accurate hand gesture prediction is crucial for effective upper-limb prosthetic limbs control. As the high flexibility and multiple degrees of freedom exhibited by human hands, there has been a growing interest in integrating deep networks with high-density surface electromyography (HD-sEMG) grids to enhance gesture recognition capabilities. However, many existing methods fall short in fully explo… ▽ More Accurate hand gesture prediction is crucial for effective upper-limb prosthetic limbs control. As the high flexibility and multiple degrees of freedom exhibited by human hands, there has been a growing interest in integrating deep networks with high-density surface electromyography (HD-sEMG) grids to enhance gesture recognition capabilities. However, many existing methods fall short in fully exploit the specific spatial topology and temporal dependencies present in HD-sEMG data. Additionally, these studies are often limited number of gestures and lack generality. Hence, this study introduces a novel gesture recognition method, named STGCN-GR, which leverages spatio-temporal graph convolution networks for HD-sEMG-based human-machine interfaces. Firstly, we construct muscle networks based on functional connectivity between channels, creating a graph representation of HD-sEMG recordings. Subsequently, a temporal convolution module is applied to capture the temporal dependences in the HD-sEMG series and a spatial graph convolution module is employed to effectively learn the intrinsic spatial topology information among distinct HD-sEMG channels. We evaluate our proposed model on a public HD-sEMG dataset comprising a substantial number of gestures (i.e., 65). Our results demonstrate the remarkable capability of the STGCN-GR method, achieving an impressive accuracy of 91.07% in predicting gestures, which surpasses state-of-the-art deep learning methods applied to the same dataset. △ Less

Submitted 1 December, 2023; originally announced December 2023.

Predicting Continuous Locomotion Modes via Multidimensional Feature Learning from sEMG

Authors: Peiwen Fu, Wenjuan Zhong, Yuyang Zhang, Wenxuan Xiong, Yuzhou Lin, Yanlong Tai, Lin Meng, Mingming Zhang

Abstract: Walking-assistive devices require adaptive control methods to ensure smooth transitions between various modes of locomotion. For this purpose, detecting human locomotion modes (e.g., level walking or stair ascent) in advance is crucial for improving the intelligence and transparency of such robotic systems. This study proposes Deep-STF, a unified end-to-end deep learning model designed for integra… ▽ More Walking-assistive devices require adaptive control methods to ensure smooth transitions between various modes of locomotion. For this purpose, detecting human locomotion modes (e.g., level walking or stair ascent) in advance is crucial for improving the intelligence and transparency of such robotic systems. This study proposes Deep-STF, a unified end-to-end deep learning model designed for integrated feature extraction in spatial, temporal, and frequency dimensions from surface electromyography (sEMG) signals. Our model enables accurate and robust continuous prediction of nine locomotion modes and 15 transitions at varying prediction time intervals, ranging from 100 to 500 ms. In addition, we introduced the concept of 'stable prediction time' as a distinct metric to quantify prediction efficiency. This term refers to the duration during which consistent and accurate predictions of mode transitions are made, measured from the time of the fifth correct prediction to the occurrence of the critical event leading to the task transition. This distinction between stable prediction time and prediction time is vital as it underscores our focus on the precision and reliability of mode transition predictions. Experimental results showcased Deep-STP's cutting-edge prediction performance across diverse locomotion modes and transitions, relying solely on sEMG data. When forecasting 100 ms ahead, Deep-STF surpassed CNN and other machine learning techniques, achieving an outstanding average prediction accuracy of 96.48%. Even with an extended 500 ms prediction horizon, accuracy only marginally decreased to 93.00%. The averaged stable prediction times for detecting next upcoming transitions spanned from 28.15 to 372.21 ms across the 100-500 ms time advances. △ Less

Submitted 13 November, 2023; originally announced November 2023.

Comments: 10 pages,7 figures

Revisiting the Knowledge Injection Frameworks

Authors: Peng Fu, Yiming Zhang, Haobo Wang, Weikang Qiu, Junbo Zhao

Abstract: In recent years, large language models (LLMs), such as GPTs, have attained great impact worldwide. However, how to adapt these LLMs to better suit the vertical domain-specific tasks by utilizing external knowledge remains not completely solved. Indeed, there have emerged a few works on this line where most of them rely on an alignment heuristic that is built to inject the corresponding knowledge t… ▽ More In recent years, large language models (LLMs), such as GPTs, have attained great impact worldwide. However, how to adapt these LLMs to better suit the vertical domain-specific tasks by utilizing external knowledge remains not completely solved. Indeed, there have emerged a few works on this line where most of them rely on an alignment heuristic that is built to inject the corresponding knowledge tuple into the associated text sample. However, despite the promise, we identify a pivotal problem in this work ubiquitously. Simply put, we find that injecting unaligned (i.e., random) knowledge tuple into the LLMs achieves comparable (and sometimes better) results than the aligned knowledge being injected. We therefore take a thorough investigation of this frustrating finding on a variety of related prior work and further provide a chain of potential interpretations for the phenomenon. Based on all that, we offer a simple remediated technique. Briefly, the core of this technique is rooted in an ideological emphasis on the pruning and purification of the external knowledge base to be injected into LLMs. At last, we show that by integrating this technique into most (if not all) knowledge injection frameworks and recent LLMs, it manages to overcome the aforementioned sanity problem and further pushes the boundary of the performance of the domain-adaptive LLMs. △ Less

Submitted 2 November, 2023; originally announced November 2023.

Comments: 9 pages, 6 figures, accepted by EMNLP 2023 Main

Knowledge Extraction and Distillation from Large-Scale Image-Text Colonoscopy Records Leveraging Large Language and Vision Models

Authors: Shuo Wang, Yan Zhu, Xiaoyuan Luo, Zhiwei Yang, Yizhe Zhang, Peiyao Fu, Manning Wang, Zhijian Song, Quanlin Li, Pinghong Zhou, Yike Guo

Abstract: The development of artificial intelligence systems for colonoscopy analysis often necessitates expert-annotated image datasets. However, limitations in dataset size and diversity impede model performance and generalisation. Image-text colonoscopy records from routine clinical practice, comprising millions of images and text reports, serve as a valuable data source, though annotating them is labour… ▽ More The development of artificial intelligence systems for colonoscopy analysis often necessitates expert-annotated image datasets. However, limitations in dataset size and diversity impede model performance and generalisation. Image-text colonoscopy records from routine clinical practice, comprising millions of images and text reports, serve as a valuable data source, though annotating them is labour-intensive. Here we leverage recent advancements in large language and vision models and propose EndoKED, a data mining paradigm for deep knowledge extraction and distillation. EndoKED automates the transformation of raw colonoscopy records into image datasets with pixel-level annotation. We validate EndoKED using multi-centre datasets of raw colonoscopy records (~1 million images), demonstrating its superior performance in training polyp detection and segmentation models. Furthermore, the EndoKED pre-trained vision backbone enables data-efficient and generalisable learning for optical biopsy, achieving expert-level performance in both retrospective and prospective validation. △ Less

Submitted 17 October, 2023; originally announced October 2023.

Enhancing cardiovascular risk prediction through AI-enabled calcium-omics

Authors: Ammar Hoori, Sadeer Al-Kindi, Tao Hu, Yingnan Song, Hao Wu, Juhwan Lee, Nour Tashtish, Pingfu Fu, Robert Gilkeson, Sanjay Rajagopalan, David L. Wilson

Abstract: Background. Coronary artery calcium (CAC) is a powerful predictor of major adverse cardiovascular events (MACE). Traditional Agatston score simply sums the calcium, albeit in a non-linear way, leaving room for improved calcification assessments that will more fully capture the extent of disease. Objective. To determine if AI methods using detailed calcification features (i.e., calcium-omics) can… ▽ More Background. Coronary artery calcium (CAC) is a powerful predictor of major adverse cardiovascular events (MACE). Traditional Agatston score simply sums the calcium, albeit in a non-linear way, leaving room for improved calcification assessments that will more fully capture the extent of disease. Objective. To determine if AI methods using detailed calcification features (i.e., calcium-omics) can improve MACE prediction. Methods. We investigated additional features of calcification including assessment of mass, volume, density, spatial distribution, territory, etc. We used a Cox model with elastic-net regularization on 2457 CT calcium score (CTCS) enriched for MACE events obtained from a large no-cost CLARIFY program (ClinicalTri-als.gov Identifier: NCT04075162). We employed sampling techniques to enhance model training. We also investigated Cox models with selected features to identify explainable high-risk characteristics. Results. Our proposed calcium-omics model with modified synthetic down sampling and up sampling gave C-index (80.5%/71.6%) and two-year AUC (82.4%/74.8%) for (80:20, training/testing), respectively (sampling was applied to the training set only). Results compared favorably to Agatston which gave C-index (71.3%/70.3%) and AUC (71.8%/68.8%), respectively. Among calcium-omics features, numbers of calcifications, LAD mass, and diffusivity (a measure of spatial distribution) were important determinants of increased risk, with dense calcification (>1000HU) associated with lower risk. The calcium-omics model reclassified 63% of MACE patients to the high risk group in a held-out test. The categorical net-reclassification index was NRI=0.153. Conclusions. AI analysis of coronary calcification can lead to improved results as compared to Agatston scoring. Our findings suggest the utility of calcium-omics in improved prediction of risk. △ Less

Submitted 23 August, 2023; originally announced August 2023.

Comments: 12 pages, 8 figures, 2 tables, 4 pages supplemental, journal paper format (under review)

Towards an induction principle for nested data types

Authors: Peng Fu, Peter Selinger

Abstract: A well-known problem in the theory of dependent types is how to handle so-called nested data types. These data types are difficult to program and to reason about in total dependently typed languages such as Agda and Coq. In particular, it is not easy to derive a canonical induction principle for such types. Working towards a solution to this problem, we introduce dependently typed folds for nested… ▽ More A well-known problem in the theory of dependent types is how to handle so-called nested data types. These data types are difficult to program and to reason about in total dependently typed languages such as Agda and Coq. In particular, it is not easy to derive a canonical induction principle for such types. Working towards a solution to this problem, we introduce dependently typed folds for nested data types. Using the nested data type Bush as a guiding example, we show how to derive its dependently typed fold and induction principle. We also discuss the relationship between dependently typed folds and the more traditional higher-order folds. △ Less

Submitted 16 June, 2023; originally announced June 2023.

Comments: 11 pages

Designer Edge States in Fractional Polarization Insulators

Authors: Wei Jie Chan, Peihao Fu, L. K. Ang, Yee Sin Ang

Abstract: We theoretically investigated the topological-protected edge states (TESs) in an anisotropic honeycomb lattice with mirror and chiral symmetries, characterized by an alternative topological invariant - fractional polarization (FP), rather than the conventional Chern number. This system termed an FP insulator is a potential platform for edge-state engineering due to its disconnected TESs. These dis… ▽ More We theoretically investigated the topological-protected edge states (TESs) in an anisotropic honeycomb lattice with mirror and chiral symmetries, characterized by an alternative topological invariant - fractional polarization (FP), rather than the conventional Chern number. This system termed an FP insulator is a potential platform for edge-state engineering due to its disconnected TESs. These disconnected and robust TESs are susceptible to perturbative chiral symmetry-breaking terms which can generate various patterns including the vanishing helical, spin-polarized, and chiral TESs. Moreover, helical and chiral TES can be achieved by the finite size effect, not possible from the aforementioned terms alone. The demonstration of these various TES in an FP-insulator offers an alternative route in designing reconfigurable two-dimensional nanoelectronic devices. △ Less

Submitted 13 June, 2023; originally announced June 2023.

Comments: 9 pages, 4 figures

Stopping power of high-density alpha-particle clusters in warm dense deuterium-tritium fuels

Authors: Z. P. Fu, Z. W. Zhang, K. Lin, D. Wu, J. Zhang

Abstract: The state of burning plasma had been achieved in inertial confinement fusion (ICF), which was regarded as a great milestone for high-gain laser fusion energy. In the burning plasma, alpha particles incident on the cryogenic (warm dense) fuels cannot be simply regarded as single particles, and the new physics brought about by the density effects of alpha particles should be considered. In this work… ▽ More The state of burning plasma had been achieved in inertial confinement fusion (ICF), which was regarded as a great milestone for high-gain laser fusion energy. In the burning plasma, alpha particles incident on the cryogenic (warm dense) fuels cannot be simply regarded as single particles, and the new physics brought about by the density effects of alpha particles should be considered. In this work, the collective interaction between them has been considered, namely the effect of the superposition of wake waves. The stopping power of alpha-particle clusters, i.e. the rate of energy loss per unit distance traveled has been calculated using both analytical and simulation approaches. In theory, we obtain the stopping power of alpha clusters in cryogenic (warm dense) fuel by the dielectric function method, which illustrates the importance of the effective interaction between particles. Simulation results using the LAPINS code show that the collective stopping power of the alpha cluster is indeed increased via coherent superposition of excitation fields (the excitation of high-amplitude wake waves). However, the comparison between simulation and theoretical results also illustrates a coherent-decoherent transition of the stopping power of the cluster. The initial conditions with various sizes and densities of the alpha clusters have been considered to verify the condition of decoherence transition. Our work provides a theoretical description of the transport properties of high-density alpha particles in warm dense cryogenic fuel and might give some theoretical guidance for the design of actual fusion processes. △ Less

Submitted 27 April, 2023; v1 submitted 27 April, 2023; originally announced April 2023.

Comments: Junior undergraduate students Z. P. Fu, Z. W. Zhang and K. Lin contributed equally to this work

Understanding Accelerated Gradient Methods: Lyapunov Analyses and Hamiltonian Assisted Interpretations

Authors: Penghui Fu, Zhiqiang Tan

Abstract: We formulate two classes of first-order algorithms more general than previously studied for minimizing smooth and strongly convex or, respectively, smooth and convex functions. We establish sufficient conditions, via new discrete Lyapunov analyses, for achieving accelerated convergence rates which match Nesterov's methods in the strongly and general convex settings. Next, we study the convergence… ▽ More We formulate two classes of first-order algorithms more general than previously studied for minimizing smooth and strongly convex or, respectively, smooth and convex functions. We establish sufficient conditions, via new discrete Lyapunov analyses, for achieving accelerated convergence rates which match Nesterov's methods in the strongly and general convex settings. Next, we study the convergence of limiting ordinary differential equations (ODEs) and point out currently notable gaps between the convergence properties of the corresponding algorithms and ODEs. Finally, we propose a novel class of discrete algorithms, called the Hamiltonian assisted gradient method, directly based on a Hamiltonian function and several interpretable operations, and then demonstrate meaningful and unified interpretations of our acceleration conditions. △ Less

Submitted 19 April, 2023; originally announced April 2023.

Visualization of Photonic Band Structures via Far-field Measurements in SiNx Photonic Crystal Slabs

Authors: Wenze Lan, Peng Fu, Chang-Yin Ji, Gang Wang, Yugui Yao, Changzhi Gu, Baoli Liu

Abstract: The band structures of the photonic crystal slabs play a significant role in manipulating the flow of light and pre-dicting exotic physics in photonics. In this letter, we show that the key features of photonic band structures can be achieved experimentally by the polarization- and momentum-resolved photoluminescence spectroscopy utilizing the light emission properties of SiNx. The two-dimensional… ▽ More The band structures of the photonic crystal slabs play a significant role in manipulating the flow of light and pre-dicting exotic physics in photonics. In this letter, we show that the key features of photonic band structures can be achieved experimentally by the polarization- and momentum-resolved photoluminescence spectroscopy utilizing the light emission properties of SiNx. The two-dimensional spectra clearly reveal the energy-momentum dispersion of band structures which is in perfect agreement with the simulation results. The isofrequency contours can be measured easily by adding a bandpass filter with a desired photon energy. Furthermore, it is convenient to observe clearly and directly the optical singularity -- the optical bound states in the continuum featured by dark point in three-dimensional photoluminescence spectra. The polarization-resolved isofrequency contours clearly show that this dark point is the center of an azimuthally polarized vortex. Finally, the helical topological edge states can be easily observed in photonic topological insulators with deformed hexagonal lattices. Our work provides a simple and effective approach for exploring topological photonics and other intriguing phenomena hidden in the photonic crystal slabs. △ Less

Submitted 18 April, 2023; v1 submitted 19 March, 2023; originally announced March 2023.

Comments: 6 pages, 5 figures

Journal ref: Appl. Phys. Lett. 122, 151102 (2023)

Field-Effect Josephson Diode via Asymmetric Spin-Momentum-Locking States

Authors: Pei-Hao Fu, Yong Xu, Shengyuan A. Yang, Ching Hua Lee, Yee Sin Ang, Jun-Feng Liu

Abstract: Recent breakthroughs in Josephson diodes dangle the possibility of extending conventional non-reciprocal electronics into the realm of superconductivity. While a strong magnetic field is recognized for enhancing diode efficiency, it concurrently poses a risk of undermining the essential superconductivity required for non-dissipative devices. To circumvent the need for magnetic-based tuning, we pro… ▽ More Recent breakthroughs in Josephson diodes dangle the possibility of extending conventional non-reciprocal electronics into the realm of superconductivity. While a strong magnetic field is recognized for enhancing diode efficiency, it concurrently poses a risk of undermining the essential superconductivity required for non-dissipative devices. To circumvent the need for magnetic-based tuning, we propose a field-effect Josephson diode based on the electrostatic gate control of finite momentum Cooper pairs in asymmetric spin-momentum-locking states. We propose two possible implementations of our gate-controlled mechanism: (i) a topological field-effect Josephson diode in time-reversal-broken quantum spin Hall insulators; and (ii) semiconductor-based field-effect Josephson diodes attainable in current experimental setups involving a Zeeman field and spin-orbit coupling. Notably, the diode efficiency is highly enhanced in the topological field-effect Josephson diode because the current carried by the asymmetric helical edge states is topologically protected and can be tuned by local gates. In the proposed Josephson diode, the combination of gates and asymmetric spin-momentum-locking nature is equivalent to that of a magnetic field, thus providing an alternative electrical operation in designing nonreciprocal superconducting devices. △ Less

Submitted 3 May, 2024; v1 submitted 4 December, 2022; originally announced December 2022.

Comments: 24 pages, 14 figures

Vision meets algae: A novel way for microalgae recognization and health monitor

Authors: Shizheng Zhou, Juntao Jiang, Xiaohan Hong, Yan Hong, Pengcheng Fu

Abstract: Marine microalgae are widespread in the ocean and play a crucial role in the ecosystem. Automatic identification and location of marine microalgae in microscopy images would help establish marine ecological environment monitoring and water quality evaluation system. We proposed a new dataset for the detection of marine microalgae and a range of detection methods, the dataset including images of di… ▽ More Marine microalgae are widespread in the ocean and play a crucial role in the ecosystem. Automatic identification and location of marine microalgae in microscopy images would help establish marine ecological environment monitoring and water quality evaluation system. We proposed a new dataset for the detection of marine microalgae and a range of detection methods, the dataset including images of different genus of algae and the same genus in different states. We set the number of unbalanced classes in the data set and added images of mixed water samples in the test set to simulate the actual situation in the field. Then we trained, validated and tested the, TOOD, YOLOv5, YOLOv8 and variants of RCNN algorithms on this dataset. The results showed both one-stage and two-stage object detection models can achieve high mean average precision, which proves the ability of computer vision in multi-object detection of microalgae, and provides basic data and models for real-time detection of microalgal cells. △ Less

Submitted 26 August, 2024; v1 submitted 14 November, 2022; originally announced November 2022.

Probing Phase Transition of Band Topology via Radiation Topology

Authors: Chang-Yin Ji, Wenze Lan, Peng Fu, Gang Wang, Changzhi Gu, Jiafang Li, Yugui Yao, Baoli Liu

Abstract: Topological photonics has received extensive attention from researchers because it provides brand new physical principles to manipulate light. Band topology of optical materials is characterized using the Berry phase defined by Bloch states. Until now, the criteria for experimentally probing the topological phase transition of band topology has always been relatively lacking in topological physics… ▽ More Topological photonics has received extensive attention from researchers because it provides brand new physical principles to manipulate light. Band topology of optical materials is characterized using the Berry phase defined by Bloch states. Until now, the criteria for experimentally probing the topological phase transition of band topology has always been relatively lacking in topological physics. Moreover, radiation topology can be aroused by the far-field polarizations of the radiating Bloch states, which is described by the Stokes phase. Although such two types of topologies are both related to Bloch states on the band structure, it is rather surprising that their development is almost independent. Here, we reveal that the phase transition of band topology can be probed by the radiation topology. We theoretically design and experimentally demonstrate such an intriguing phenomenon by constructing photonic crystals that support optical analogs of quantum spin Hall effects. The results show that the topological charge of the far-field polarization vortex changes from +1 to -2 or from -2 to +1 when the band topology changes from trivial to non-trivial, which provides a new criterion to probe the phase transition of band topology using radiation topology. Our findings not only provide an insightful understanding of band topology and radiation topology, but also can serve as a novel route to manipulate the near and far fields of light. △ Less

Submitted 31 October, 2022; originally announced October 2022.

Potential scientific synergies in weak lensing studies between the CSST and Euclid space probes

Authors: D. Z. Liu, X. M. Meng, X. Z. Er, Z. H. Fan, M. Kilbinger, G. L. Li, R. Li, T. Schrabback, D. Scognamiglio, H. Y. Shan, C. Tao, Y. S. Ting, J. Zhang, S. H. Cheng, S. Farrens, L. P. Fu, H. Hildebrandt, X. Kang, J. P. Kneib, X. K. Liu, Y. Mellier, R. Nakajima, P. Schneider, J. L. Starck, C. L. Wei , et al. (2 additional authors not shown)

Abstract: Aims. With the next generation of large surveys coming to the stage of observational cosmology soon, it is important to explore their potential synergies and to maximise their scientific outcomes. In this study, we aim to investigate the complementarity of the two upcoming space missions Euclid and the China Space Station Telescope (CSST), focusing on weak lensing (WL) cosmology. In particular, we… ▽ More Aims. With the next generation of large surveys coming to the stage of observational cosmology soon, it is important to explore their potential synergies and to maximise their scientific outcomes. In this study, we aim to investigate the complementarity of the two upcoming space missions Euclid and the China Space Station Telescope (CSST), focusing on weak lensing (WL) cosmology. In particular, we analyse the photometric redshifts (photo-zs) and the galaxy blending effects. For Euclid, WL measurements suffer from chromatic PSF effects. For this, CSST can provide valuable information for Euclid to obtain more accurate PSF, and to calibrate the color and color-gradient biases for WL measurements. Methods. We create image simulations for different surveys, and quantify the photo-z performance. For blending analyses, we employ high-resolution HST/CANDELS data to mock Euclid, CSST, and an LSST-like survey. We analyse the blending fraction for different cases, and the blending effects on galaxy photometry. Furthermore, we demonstrate that CSST can provide a large enough number of high SNR multi-band galaxy images to calibrate the color-gradient biases for Euclid. Results. The sky coverage of Euclid lies entirely within the CSST footprint. The combination of Euclid with CSST data can be done more uniformly than with the various ground-based data. Our studies show that by combining Euclid and CSST, we can reach a photo-z precision of $σ_{\rm NMAD} \approx 0.04$, and an outlier fraction of $η\approx 2.4\%$. Because of the similarly high resolutions, the data combination of Euclid and CSST can be relatively straightforward for photometry. To include ground-based data, however, sophisticated deblending utilizing priors from high-resolution space data is demanded. The color-gradient biases for Euclid can be well calibrated to the level of 0.1% using galaxies from CSST deep survey. △ Less

Submitted 28 October, 2022; originally announced October 2022.

Comments: 18 pages, 19 figures and 2 tables. Accepted for publication in A&A

Journal ref: A&A 669, A128 (2023)

Compressing And Debiasing Vision-Language Pre-Trained Models for Visual Question Answering

Authors: Qingyi Si, Yuanxin Liu, Zheng Lin, Peng Fu, Weiping Wang

Abstract: Despite the excellent performance of vision-language pre-trained models (VLPs) on conventional VQA task, they still suffer from two problems: First, VLPs tend to rely on language biases in datasets and fail to generalize to out-of-distribution (OOD) data. Second, they are inefficient in terms of memory footprint and computation. Although promising progress has been made in both problems, most exis… ▽ More Despite the excellent performance of vision-language pre-trained models (VLPs) on conventional VQA task, they still suffer from two problems: First, VLPs tend to rely on language biases in datasets and fail to generalize to out-of-distribution (OOD) data. Second, they are inefficient in terms of memory footprint and computation. Although promising progress has been made in both problems, most existing works tackle them independently. To facilitate the application of VLP to VQA tasks, it is imperative to jointly study VLP compression and OOD robustness, which, however, has not yet been explored. This paper investigates whether a VLP can be compressed and debiased simultaneously by searching sparse and robust subnetworks. To this end, we systematically study the design of a training and compression pipeline to search the subnetworks, as well as the assignment of sparsity to different modality-specific modules. Our experiments involve 3 VLPs, 2 compression methods, 4 training methods, 2 datasets and a range of sparsity levels and random seeds. Our results show that there indeed exist sparse and robust subnetworks, which are competitive with the debiased full VLP and clearly outperform the debiasing SoTAs with fewer parameters on OOD datasets VQA-CP v2 and VQA-VS. The codes can be found at https://github.com/PhoebusSi/Compress-Robust-VQA. △ Less

Submitted 11 October, 2023; v1 submitted 26 October, 2022; originally announced October 2022.

Comments: EMNLP 2023

Question-Interlocutor Scope Realized Graph Modeling over Key Utterances for Dialogue Reading Comprehension

Authors: Jiangnan Li, Mo Yu, Fandong Meng, Zheng Lin, Peng Fu, Weiping Wang, Jie Zhou

Abstract: In this work, we focus on dialogue reading comprehension (DRC), a task extracting answer spans for questions from dialogues. Dialogue context modeling in DRC is tricky due to complex speaker information and noisy dialogue context. To solve the two problems, previous research proposes two self-supervised tasks respectively: guessing who a randomly masked speaker is according to the dialogue and pre… ▽ More In this work, we focus on dialogue reading comprehension (DRC), a task extracting answer spans for questions from dialogues. Dialogue context modeling in DRC is tricky due to complex speaker information and noisy dialogue context. To solve the two problems, previous research proposes two self-supervised tasks respectively: guessing who a randomly masked speaker is according to the dialogue and predicting which utterance in the dialogue contains the answer. Although these tasks are effective, there are still urging problems: (1) randomly masking speakers regardless of the question cannot map the speaker mentioned in the question to the corresponding speaker in the dialogue, and ignores the speaker-centric nature of utterances. This leads to wrong answer extraction from utterances in unrelated interlocutors' scopes; (2) the single utterance prediction, preferring utterances similar to the question, is limited in finding answer-contained utterances not similar to the question. To alleviate these problems, we first propose a new key utterances extracting method. It performs prediction on the unit formed by several contiguous utterances, which can realize more answer-contained utterances. Based on utterances in the extracted units, we then propose Question-Interlocutor Scope Realized Graph (QuISG) modeling. As a graph constructed on the text of utterances, QuISG additionally involves the question and question-mentioning speaker names as nodes. To realize interlocutor scopes, speakers in the dialogue are connected with the words in their corresponding utterances. Experiments on the benchmarks show that our method can achieve better and competitive results against previous works. △ Less

Submitted 26 October, 2022; originally announced October 2022.

Block-wise Primal-dual Algorithms for Large-scale Doubly Penalized ANOVA Modeling

Authors: Penghui Fu, Zhiqiang Tan

Abstract: For multivariate nonparametric regression, doubly penalized ANOVA modeling (DPAM) has recently been proposed, using hierarchical total variations (HTVs) and empirical norms as penalties on the component functions such as main effects and multi-way interactions in a functional ANOVA decomposition of the underlying regression function. The two penalties play complementary roles: the HTV penalty prom… ▽ More For multivariate nonparametric regression, doubly penalized ANOVA modeling (DPAM) has recently been proposed, using hierarchical total variations (HTVs) and empirical norms as penalties on the component functions such as main effects and multi-way interactions in a functional ANOVA decomposition of the underlying regression function. The two penalties play complementary roles: the HTV penalty promotes sparsity in the selection of basis functions within each component function, whereas the empirical-norm penalty promotes sparsity in the selection of component functions. We adopt backfitting or block minimization for training DPAM, and develop two suitable primal-dual algorithms, including both batch and stochastic versions, for updating each component function in single-block optimization. Existing applications of primal-dual algorithms are intractable in our setting with both HTV and empirical-norm penalties. Through extensive numerical experiments, we demonstrate the validity and advantage of our stochastic primal-dual algorithms, compared with their batch versions and a previous active-set algorithm, in large-scale scenarios. △ Less

Submitted 19 October, 2022; originally announced October 2022.

A Win-win Deal: Towards Sparse and Robust Pre-trained Language Models

Authors: Yuanxin Liu, Fandong Meng, Zheng Lin, Jiangnan Li, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Abstract: Despite the remarkable success of pre-trained language models (PLMs), they still face two challenges: First, large-scale PLMs are inefficient in terms of memory footprint and computation. Second, on the downstream tasks, PLMs tend to rely on the dataset bias and struggle to generalize to out-of-distribution (OOD) data. In response to the efficiency problem, recent studies show that dense PLMs can… ▽ More Despite the remarkable success of pre-trained language models (PLMs), they still face two challenges: First, large-scale PLMs are inefficient in terms of memory footprint and computation. Second, on the downstream tasks, PLMs tend to rely on the dataset bias and struggle to generalize to out-of-distribution (OOD) data. In response to the efficiency problem, recent studies show that dense PLMs can be replaced with sparse subnetworks without hurting the performance. Such subnetworks can be found in three scenarios: 1) the fine-tuned PLMs, 2) the raw PLMs and then fine-tuned in isolation, and even inside 3) PLMs without any parameter fine-tuning. However, these results are only obtained in the in-distribution (ID) setting. In this paper, we extend the study on PLMs subnetworks to the OOD setting, investigating whether sparsity and robustness to dataset bias can be achieved simultaneously. To this end, we conduct extensive experiments with the pre-trained BERT model on three natural language understanding (NLU) tasks. Our results demonstrate that \textbf{sparse and robust subnetworks (SRNets) can consistently be found in BERT}, across the aforementioned three scenarios, using different training and compression methods. Furthermore, we explore the upper bound of SRNets using the OOD information and show that \textbf{there exist sparse and almost unbiased BERT subnetworks}. Finally, we present 1) an analytical study that provides insights on how to promote the efficiency of SRNets searching process and 2) a solution to improve subnetworks' performance at high sparsity. The code is available at https://github.com/llyx97/sparse-and-robust-PLM. △ Less

Submitted 11 October, 2022; originally announced October 2022.

Comments: Accepted by NeurIPS 2022

Language Prior Is Not the Only Shortcut: A Benchmark for Shortcut Learning in VQA

Authors: Qingyi Si, Fandong Meng, Mingyu Zheng, Zheng Lin, Yuanxin Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Abstract: Visual Question Answering (VQA) models are prone to learn the shortcut solution formed by dataset biases rather than the intended solution. To evaluate the VQA models' reasoning ability beyond shortcut learning, the VQA-CP v2 dataset introduces a distribution shift between the training and test set given a question type. In this way, the model cannot use the training set shortcut (from question ty… ▽ More Visual Question Answering (VQA) models are prone to learn the shortcut solution formed by dataset biases rather than the intended solution. To evaluate the VQA models' reasoning ability beyond shortcut learning, the VQA-CP v2 dataset introduces a distribution shift between the training and test set given a question type. In this way, the model cannot use the training set shortcut (from question type to answer) to perform well on the test set. However, VQA-CP v2 only considers one type of shortcut and thus still cannot guarantee that the model relies on the intended solution rather than a solution specific to this shortcut. To overcome this limitation, we propose a new dataset that considers varying types of shortcuts by constructing different distribution shifts in multiple OOD test sets. In addition, we overcome the three troubling practices in the use of VQA-CP v2, e.g., selecting models using OOD test sets, and further standardize OOD evaluation procedure. Our benchmark provides a more rigorous and comprehensive testbed for shortcut learning in VQA. We benchmark recent methods and find that methods specifically designed for particular shortcuts fail to simultaneously generalize to our varying OOD test sets. We also systematically study the varying shortcuts and provide several valuable findings, which may promote the exploration of shortcut learning in VQA. △ Less

Submitted 10 October, 2022; originally announced October 2022.

Comments: Fingdings of EMNLP-2022

Towards Robust Visual Question Answering: Making the Most of Biased Samples via Contrastive Learning

Authors: Qingyi Si, Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Abstract: Models for Visual Question Answering (VQA) often rely on the spurious correlations, i.e., the language priors, that appear in the biased samples of training set, which make them brittle against the out-of-distribution (OOD) test data. Recent methods have achieved promising progress in overcoming this problem by reducing the impact of biased samples on model training. However, these models reveal a… ▽ More Models for Visual Question Answering (VQA) often rely on the spurious correlations, i.e., the language priors, that appear in the biased samples of training set, which make them brittle against the out-of-distribution (OOD) test data. Recent methods have achieved promising progress in overcoming this problem by reducing the impact of biased samples on model training. However, these models reveal a trade-off that the improvements on OOD data severely sacrifice the performance on the in-distribution (ID) data (which is dominated by the biased samples). Therefore, we propose a novel contrastive learning approach, MMBS, for building robust VQA models by Making the Most of Biased Samples. Specifically, we construct positive samples for contrastive learning by eliminating the information related to spurious correlation from the original training samples and explore several strategies to use the constructed positive samples for training. Instead of undermining the importance of biased samples in model training, our approach precisely exploits the biased samples for unbiased information that contributes to reasoning. The proposed method is compatible with various VQA backbones. We validate our contributions by achieving competitive performance on the OOD dataset VQA-CP v2 while preserving robust performance on the ID dataset VQA v2. △ Less

Submitted 10 October, 2022; originally announced October 2022.

Comments: Findings of EMNLP-2022

Novel Radiomic Measurements of Tumor- Associated Vasculature Morphology on Clinical Imaging as a Biomarker of Treatment Response in Multiple Cancers

Authors: Nathaniel Braman, Prateek Prasanna, Kaustav Bera, Mehdi Alilou, Mohammadhadi Khorrami, Patrick Leo, Maryam Etesami, Manasa Vulchi, Paulette Turk, Amit Gupta, Prantesh Jain, Pingfu Fu, Nathan Pennell, Vamsidhar Velcheti, Jame Abraham, Donna Plecha, Anant Madabhushi

Abstract: Purpose: Tumor-associated vasculature differs from healthy blood vessels by its chaotic architecture and twistedness, which promotes treatment resistance. Measurable differences in these attributes may help stratify patients by likely benefit of systemic therapy (e.g. chemotherapy). In this work, we present a new category of radiomic biomarkers called quantitative tumor-associated vasculature (Qua… ▽ More Purpose: Tumor-associated vasculature differs from healthy blood vessels by its chaotic architecture and twistedness, which promotes treatment resistance. Measurable differences in these attributes may help stratify patients by likely benefit of systemic therapy (e.g. chemotherapy). In this work, we present a new category of radiomic biomarkers called quantitative tumor-associated vasculature (QuanTAV) features, and demonstrate their ability to predict response and survival across multiple cancers, imaging modalities, and treatment regimens. Experimental Design: We segmented tumor vessels and computed mathematical measurements of twistedness and organization on routine pre-treatment radiology (CT or contrast-enhanced MRI) from 558 patients, who received one of four first-line chemotherapy-based therapeutic intervention strategies for breast (n=371) or non-small cell lung cancer (NSCLC, n=187). Results: Across 4 chemotherapy-based treatment strategies, classifiers of QuanTAV measurements significantly (p<.05) predicted response in held out testing cohorts alone (AUC=0.63-0.71) and increased AUC by 0.06-0.12 when added to models of significant clinical variables alone. QuanTAV risk scores were prognostic of recurrence free survival in treatment cohorts chemotherapy for breast cancer (p=0.002, HR=1.25, 95% CI 1.08-1.44, C-index=.66) and chemoradiation for NSCLC (p=0.039, HR=1.28, 95% CI 1.01-1.62, C-index=0.66). Categorical QuanTAV risk groups were independently prognostic among all treatment groups, including NSCLC patients receiving chemotherapy (p=0.034, HR=2.29, 95% CI 1.07-4.94, C-index=0.62). Conclusions: Across these domains, we observed an association of vascular morphology on radiology with treatment outcome. Our findings suggest the potential of tumor-associated vasculature shape and structure as a prognostic and predictive biomarker for multiple cancers and treatments. △ Less

Submitted 5 October, 2022; originally announced October 2022.

Comments: This manuscript has been accepted for publication in Clinical Cancer Research, which is published by the American Association for Cancer Research

Transfer Learning and Vision Transformer based State-of-Health prediction of Lithium-Ion Batteries

Authors: Pengyu Fu, Liang Chu, Zhuoran Hou, Jincheng Hu, Yanjun Huang, Yuanjian Zhang

Abstract: In recent years, significant progress has been made in transportation electrification. And lithium-ion batteries (LIB), as the main energy storage devices, have received widespread attention. Accurately predicting the state of health (SOH) can not only ease the anxiety of users about the battery life but also provide important information for the management of the battery. This paper presents a pr… ▽ More In recent years, significant progress has been made in transportation electrification. And lithium-ion batteries (LIB), as the main energy storage devices, have received widespread attention. Accurately predicting the state of health (SOH) can not only ease the anxiety of users about the battery life but also provide important information for the management of the battery. This paper presents a prediction method for SOH based on Vision Transformer (ViT) model. First, discrete charging data of a predefined voltage range is used as an input data matrix. Then, the cycle features of the battery are captured by the ViT which can obtain the global features, and the SOH is obtained by combining the cycle features with the full connection (FC) layer. At the same time, transfer learning (TL) is introduced, and the prediction model based on source task battery training is further fine-tuned according to the early cycle data of the target task battery to provide an accurate prediction. Experiments show that our method can obtain better feature expression compared with existing deep learning methods so that better prediction effect and transfer effect can be achieved. △ Less

Submitted 7 September, 2022; originally announced September 2022.

Comments: 13 pages,15 figures,13 equations

A Transferable Intersection Reconstruction Network for Traffic Speed Prediction

Authors: Pengyu Fu, Liang Chu, Zhuoran Hou, Jincheng Hu, Yanjun Huang, Yuanjian Zhang

Abstract: Traffic speed prediction is the key to many valuable applications, and it is also a challenging task because of its various influencing factors. Recent work attempts to obtain more information through various hybrid models, thereby improving the prediction accuracy. However, the spatial information acquisition schemes of these methods have two-level differentiation problems. Either the modeling is… ▽ More Traffic speed prediction is the key to many valuable applications, and it is also a challenging task because of its various influencing factors. Recent work attempts to obtain more information through various hybrid models, thereby improving the prediction accuracy. However, the spatial information acquisition schemes of these methods have two-level differentiation problems. Either the modeling is simple but contains little spatial information, or the modeling is complete but lacks flexibility. In order to introduce more spatial information on the basis of ensuring flexibility, this paper proposes IRNet (Transferable Intersection Reconstruction Network). First, this paper reconstructs the intersection into a virtual intersection with the same structure, which simplifies the topology of the road network. Then, the spatial information is subdivided into intersection information and sequence information of traffic flow direction, and spatiotemporal features are obtained through various models. Third, a self-attention mechanism is used to fuse spatiotemporal features for prediction. In the comparison experiment with the baseline, not only the prediction effect, but also the transfer performance has obvious advantages. △ Less

Submitted 22 July, 2022; originally announced July 2022.

Comments: 14 pages, 12 figures

On the Lambek embedding and the category of product-preserving presheaves

Authors: Peng Fu, Kohei Kishida, Neil J. Ross, Peter Selinger

Abstract: It is well-known that the category of presheaf functors is complete and cocomplete, and that the Yoneda embedding into the presheaf category preserves products. However, the Yoneda embedding does not preserve coproducts. It is perhaps less well-known that if we restrict the codomain of the Yoneda embedding to the full subcategory of limit-preserving functors, then this embedding preserves colimits… ▽ More It is well-known that the category of presheaf functors is complete and cocomplete, and that the Yoneda embedding into the presheaf category preserves products. However, the Yoneda embedding does not preserve coproducts. It is perhaps less well-known that if we restrict the codomain of the Yoneda embedding to the full subcategory of limit-preserving functors, then this embedding preserves colimits, while still enjoying most of the other useful properties of the Yoneda embedding. We call this modified embedding the Lambek embedding. The category of limit-preserving functors is known to be a reflective subcategory of the category of all functors, i.e., there is a left adjoint for the inclusion functor. In the literature, the existence of this left adjoint is often proved non-constructively, e.g., by an application of Freyd's adjoint functor theorem. In this paper, we provide an alternative, more constructive proof of this fact. We first explain the Lambek embedding and why it preserves coproducts. Then we review some concepts from multi-sorted algebras and observe that there is a one-to-one correspondence between product-preserving presheaves and certain multi-sorted term algebras. We provide a construction that freely turns any presheaf functor into a product-preserving one, hence giving an explicit definition of the left adjoint functor of the inclusion. Finally, we sketch how to extend our method to prove that the subcategory of limit-preserving functors is also reflective. △ Less

Submitted 12 May, 2022; originally announced May 2022.

Neutral Utterances are Also Causes: Enhancing Conversational Causal Emotion Entailment with Social Commonsense Knowledge

Authors: Jiangnan Li, Fandong Meng, Zheng Lin, Rui Liu, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Abstract: Conversational Causal Emotion Entailment aims to detect causal utterances for a non-neutral targeted utterance from a conversation. In this work, we build conversations as graphs to overcome implicit contextual modelling of the original entailment style. Following the previous work, we further introduce the emotion information into graphs. Emotion information can markedly promote the detection of… ▽ More Conversational Causal Emotion Entailment aims to detect causal utterances for a non-neutral targeted utterance from a conversation. In this work, we build conversations as graphs to overcome implicit contextual modelling of the original entailment style. Following the previous work, we further introduce the emotion information into graphs. Emotion information can markedly promote the detection of causal utterances whose emotion is the same as the targeted utterance. However, it is still hard to detect causal utterances with different emotions, especially neutral ones. The reason is that models are limited in reasoning causal clues and passing them between utterances. To alleviate this problem, we introduce social commonsense knowledge (CSK) and propose a Knowledge Enhanced Conversation graph (KEC). KEC propagates the CSK between two utterances. As not all CSK is emotionally suitable for utterances, we therefore propose a sentiment-realized knowledge selecting strategy to filter CSK. To process KEC, we further construct the Knowledge Enhanced Directed Acyclic Graph networks. Experimental results show that our method outperforms baselines and infers more causes with different emotions from the targeted utterance. △ Less

Submitted 7 May, 2022; v1 submitted 2 May, 2022; originally announced May 2022.

Proto-Quipper with dynamic lifting

Authors: Peng Fu, Kohei Kishida, Neil J. Ross, Peter Selinger

Abstract: Quipper is a functional programming language for quantum computing. Proto-Quipper is a family of languages aiming to provide a formal foundation for Quipper. In this paper, we extend Proto-Quipper-M with a construct called dynamic lifting, which is present in Quipper. By virtue of being a circuit description language, Proto-Quipper has two separate runtimes: circuit generation time and circuit exe… ▽ More Quipper is a functional programming language for quantum computing. Proto-Quipper is a family of languages aiming to provide a formal foundation for Quipper. In this paper, we extend Proto-Quipper-M with a construct called dynamic lifting, which is present in Quipper. By virtue of being a circuit description language, Proto-Quipper has two separate runtimes: circuit generation time and circuit execution time. Values that are known at circuit generation time are called parameters, and values that are known at circuit execution time are called states. Dynamic lifting is an operation that enables a state, such as the result of a measurement, to be lifted to a parameter, where it can influence the generation of the next portion of the circuit. As a result, dynamic lifting enables Proto-Quipper programs to interleave classical and quantum computation. We describe the syntax of a language we call Proto-Quipper-Dyn. Its type system uses a system of modalities to keep track of the use of dynamic lifting. We also provide an operational semantics, as well as an abstract categorical semantics for dynamic lifting based on enriched category theory. We prove that both the type system and the operational semantics are sound with respect to our categorical semantics. Finally, we give some examples of Proto-Quipper-Dyn programs that make essential use of dynamic lifting. △ Less

Submitted 8 November, 2022; v1 submitted 27 April, 2022; originally announced April 2022.

A Biset-Enriched Categorical Model for Proto-Quipper with Dynamic Lifting

Authors: Peng Fu, Kohei Kishida, Neil J. Ross, Peter Selinger

Abstract: Quipper and Proto-Quipper are a family of quantum programming languages that, by their nature as circuit description languages, involve two runtimes: one at which the program generates a circuit and one at which the circuit is executed, normally with probabilistic results due to measurements. Accordingly, the language distinguishes two kinds of data: parameters, which are known at circuit generati… ▽ More Quipper and Proto-Quipper are a family of quantum programming languages that, by their nature as circuit description languages, involve two runtimes: one at which the program generates a circuit and one at which the circuit is executed, normally with probabilistic results due to measurements. Accordingly, the language distinguishes two kinds of data: parameters, which are known at circuit generation time, and states, which are known at circuit execution time. Sometimes, it is desirable for the results of measurements to control the generation of the next part of the circuit. Therefore, the language needs to turn states, such as measurement outcomes, into parameters, an operation we call dynamic lifting. The goal of this paper is to model this interaction between the runtimes by providing a general categorical structure enriched in what we call "bisets". We demonstrate that the biset-enriched structure achieves a proper semantics of the two runtimes and their interaction, by showing that it models a variant of Proto-Quipper with dynamic lifting. The present paper deals with the concrete categorical semantics of this language, whereas a companion paper deals with the syntax, type system, operational semantics, and abstract categorical semantics. △ Less

Submitted 15 November, 2023; v1 submitted 27 April, 2022; originally announced April 2022.

Comments: In Proceedings QPL 2022, arXiv:2311.08375

Journal ref: EPTCS 394, 2023, pp. 302-342

Learning to Win Lottery Tickets in BERT Transfer via Task-agnostic Mask Training

Authors: Yuanxin Liu, Fandong Meng, Zheng Lin, Peng Fu, Yanan Cao, Weiping Wang, Jie Zhou

Abstract: Recent studies on the lottery ticket hypothesis (LTH) show that pre-trained language models (PLMs) like BERT contain matching subnetworks that have similar transfer learning performance as the original PLM. These subnetworks are found using magnitude-based pruning. In this paper, we find that the BERT subnetworks have even more potential than these studies have shown. Firstly, we discover that the… ▽ More Recent studies on the lottery ticket hypothesis (LTH) show that pre-trained language models (PLMs) like BERT contain matching subnetworks that have similar transfer learning performance as the original PLM. These subnetworks are found using magnitude-based pruning. In this paper, we find that the BERT subnetworks have even more potential than these studies have shown. Firstly, we discover that the success of magnitude pruning can be attributed to the preserved pre-training performance, which correlates with the downstream transferability. Inspired by this, we propose to directly optimize the subnetwork structure towards the pre-training objectives, which can better preserve the pre-training performance. Specifically, we train binary masks over model weights on the pre-training tasks, with the aim of preserving the universal transferability of the subnetwork, which is agnostic to any specific downstream tasks. We then fine-tune the subnetworks on the GLUE benchmark and the SQuAD dataset. The results show that, compared with magnitude pruning, mask training can effectively find BERT subnetworks with improved overall performance on downstream tasks. Moreover, our method is also more efficient in searching subnetworks and more advantageous when fine-tuning within a certain range of data scarcity. Our code is available at https://github.com/llyx97/TAMT. △ Less

Submitted 29 May, 2022; v1 submitted 24 April, 2022; originally announced April 2022.

Comments: Accepted by NAACL 2022

6GAN: IPv6 Multi-Pattern Target Generation via Generative Adversarial Nets with Reinforcement Learning

Authors: Tianyu Cui, Gaopeng Gou, Gang Xiong, Chang Liu, Peipei Fu, Zhen Li

Abstract: Global IPv6 scanning has always been a challenge for researchers because of the limited network speed and computational power. Target generation algorithms are recently proposed to overcome the problem for Internet assessments by predicting a candidate set to scan. However, IPv6 custom address configuration emerges diverse addressing patterns discouraging algorithmic inference. Widespread IPv6 ali… ▽ More Global IPv6 scanning has always been a challenge for researchers because of the limited network speed and computational power. Target generation algorithms are recently proposed to overcome the problem for Internet assessments by predicting a candidate set to scan. However, IPv6 custom address configuration emerges diverse addressing patterns discouraging algorithmic inference. Widespread IPv6 alias could also mislead the algorithm to discover aliased regions rather than valid host targets. In this paper, we introduce 6GAN, a novel architecture built with Generative Adversarial Net (GAN) and reinforcement learning for multi-pattern target generation. 6GAN forces multiple generators to train with a multi-class discriminator and an alias detector to generate non-aliased active targets with different addressing pattern types. The rewards from the discriminator and the alias detector help supervise the address sequence decision-making process. After adversarial training, 6GAN's generators could keep a strong imitating ability for each pattern and 6GAN's discriminator obtains outstanding pattern discrimination ability with a 0.966 accuracy. Experiments indicate that our work outperformed the state-of-the-art target generation algorithms by reaching a higher-quality candidate set. △ Less

Submitted 20 April, 2022; originally announced April 2022.

Comments: The paper has been accepted at the 2021 IEEE International Conference on Computer Communications (INFOCOM 2021). The source code has been published at https://github.com/CuiTianyu961030/6GAN

Deep Learning-Accelerated 3D Carbon Storage Reservoir Pressure Forecasting Based on Data Assimilation Using Surface Displacement from InSAR

Authors: Hewei Tang, Pengcheng Fu, Honggeun Jo, Su Jiang, Christopher S. Sherman, François Hamon, Nicholas A. Azzolina, Joseph P. Morris

Abstract: Fast forecasting of reservoir pressure distribution in geologic carbon storage (GCS) by assimilating monitoring data is a challenging problem. Due to high drilling cost, GCS projects usually have spatially sparse measurements from wells, leading to high uncertainties in reservoir pressure prediction. To address this challenge, we propose to use low-cost Interferometric Synthetic-Aperture Radar (In… ▽ More Fast forecasting of reservoir pressure distribution in geologic carbon storage (GCS) by assimilating monitoring data is a challenging problem. Due to high drilling cost, GCS projects usually have spatially sparse measurements from wells, leading to high uncertainties in reservoir pressure prediction. To address this challenge, we propose to use low-cost Interferometric Synthetic-Aperture Radar (InSAR) data as monitoring data to infer reservoir pressure build up. We develop a deep learning-accelerated workflow to assimilate surface displacement maps interpreted from InSAR and to forecast dynamic reservoir pressure. Employing an Ensemble Smoother Multiple Data Assimilation (ES-MDA) framework, the workflow updates three-dimensional (3D) geologic properties and predicts reservoir pressure with quantified uncertainties. We use a synthetic commercial-scale GCS model with bimodally distributed permeability and porosity to demonstrate the efficacy of the workflow. A two-step CNN-PCA approach is employed to parameterize the bimodal fields. The computational efficiency of the workflow is boosted by two residual U-Net based surrogate models for surface displacement and reservoir pressure predictions, respectively. The workflow can complete data assimilation and reservoir pressure forecasting in half an hour on a personal computer. △ Less

Submitted 26 January, 2022; v1 submitted 21 January, 2022; originally announced January 2022.

High order discontinuous cut finite element methods for linear hyperbolic conservation laws with an interface

Authors: Pei Fu, Thomas Frachon, Gunilla Kreiss, Sara Zahedi

Abstract: We develop a family of cut finite element methods of different orders based on the discontinuous Galerkin framework, for hyperbolic conservation laws with stationary interfaces in both one and two space dimensions, and for moving interfaces in one space dimension. Interface conditions are imposed weakly and so that both conservation and stability are ensured. A CutFEM with discontinuous elements i… ▽ More We develop a family of cut finite element methods of different orders based on the discontinuous Galerkin framework, for hyperbolic conservation laws with stationary interfaces in both one and two space dimensions, and for moving interfaces in one space dimension. Interface conditions are imposed weakly and so that both conservation and stability are ensured. A CutFEM with discontinuous elements in space is developed and coupled to standard explicit time-stepping schemes for linear advection problems and the acoustic wave problem with stationary interfaces. In the case of moving interfaces, we propose a space-time CutFEM based on discontinuous elements both in space and time for linear advection problems. We show that the proposed CutFEM are conservative and energy stable. For the stationary interface case an a priori error estimate is proven. Numerical computations in both one and two space dimensions support the analysis, and in addition demonstrate that the proposed methods have the expected accuracy. △ Less

Submitted 18 January, 2022; originally announced January 2022.

Machine learning-based porosity estimation from spectral decomposed seismic data

Authors: Honggeun Jo, Yongchae Cho, Michael J. Pyrcz, Hewei Tang, Pengcheng Fu

Abstract: Estimating porosity models via seismic data is challenging due to the signal noise and insufficient resolution of seismic data. Although impedance inversion is often used by combining with well logs, several hurdles remain to retrieve sub-seismic scale porosity. As an alternative, we propose a machine learning-based workflow to convert seismic data to porosity models. A ResUNet++ based workflow is… ▽ More Estimating porosity models via seismic data is challenging due to the signal noise and insufficient resolution of seismic data. Although impedance inversion is often used by combining with well logs, several hurdles remain to retrieve sub-seismic scale porosity. As an alternative, we propose a machine learning-based workflow to convert seismic data to porosity models. A ResUNet++ based workflow is designed to take three seismic data in different frequencies (i.e., decomposed seismic data) and estimate their corresponding porosity model. The workflow is successfully demonstrated in the 3D channelized reservoir to estimate the porosity model with more than 0.9 in R2 score for training and validating data. Moreover, the application is extended for a stress test by adding signal noise to the seismic data, and the workflow results show a robust estimation even with 5\% of noise. Another two ResUNet++ are trained to take either the lowest or highest resolution seismic data only to estimate the porosity model, but they show under- and over-fitting results, supporting the importance of using decomposed seismic data in porosity estimation. △ Less

Submitted 22 November, 2021; originally announced November 2021.

ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data

Authors: Gilad Baruch, Zhuoyuan Chen, Afshin Dehghan, Tal Dimry, Yuri Feigin, Peter Fu, Thomas Gebauer, Brandon Joffe, Daniel Kurz, Arik Schwartz, Elad Shulman

Abstract: Scene understanding is an active research area. Commercial depth sensors, such as Kinect, have enabled the release of several RGB-D datasets over the past few years which spawned novel methods in 3D scene understanding. More recently with the launch of the LiDAR sensor in Apple's iPads and iPhones, high quality RGB-D data is accessible to millions of people on a device they commonly use. This open… ▽ More Scene understanding is an active research area. Commercial depth sensors, such as Kinect, have enabled the release of several RGB-D datasets over the past few years which spawned novel methods in 3D scene understanding. More recently with the launch of the LiDAR sensor in Apple's iPads and iPhones, high quality RGB-D data is accessible to millions of people on a device they commonly use. This opens a whole new era in scene understanding for the Computer Vision community as well as app developers. The fundamental research in scene understanding together with the advances in machine learning can now impact people's everyday experiences. However, transforming these scene understanding methods to real-world experiences requires additional innovation and development. In this paper we introduce ARKitScenes. It is not only the first RGB-D dataset that is captured with a now widely available depth sensor, but to our best knowledge, it also is the largest indoor scene understanding data released. In addition to the raw and processed data from the mobile device, ARKitScenes includes high resolution depth maps captured using a stationary laser scanner, as well as manually labeled 3D oriented bounding boxes for a large taxonomy of furniture. We further analyze the usefulness of the data for two downstream tasks: 3D object detection and color-guided depth upsampling. We demonstrate that our dataset can help push the boundaries of existing state-of-the-art methods and it introduces new challenges that better represent real-world scenarios. △ Less

Submitted 12 January, 2022; v1 submitted 16 November, 2021; originally announced November 2021.

Transverse Hydraulic Fracture Initiation: Insights from Intermediate-Scale Hydraulic Fracture Tests

Authors: Jeffrey Burghardt, Hunter Knox, Chris Strickland, Vincent Vermeul, Pengcheng Fu, Craig Ulrich, Mark McClure, Mathew Ingraham, Paul Schwering, Timothy Kneafsey, Doug Blankenship

Abstract: In a uniform stress field, a tensile hydraulic fracture will favor propagation in the plane formed by the maximum (most compressive) and intermediate principal stresses. For the propagation of hydraulic fractures in non-uniform stress fields, especially those where the principal stress directions change spatially, things become much more complex, are less well understood, and therefore are more di… ▽ More In a uniform stress field, a tensile hydraulic fracture will favor propagation in the plane formed by the maximum (most compressive) and intermediate principal stresses. For the propagation of hydraulic fractures in non-uniform stress fields, especially those where the principal stress directions change spatially, things become much more complex, are less well understood, and therefore are more difficult to predict. Such is often the case when a hydraulic fracture is initiated from a borehole drilled in the direction of the minimum principal stress. This is a well-known problem that has been the subject of several numerical and experimental studies. The purpose of this paper is to offer new insights from a series of hydraulic fracture tests conducted in sub-horizontal boreholes drilled in the Sanford Underground Research Facility. This project, called the Enhanced Geothermal Systems (EGS) Collab project, has developed a test bed at a depth of approximately 1478 meters underground consisting of sub-horizontal injection, production, and monitoring wells. This paper presents stimulation test data, and supporting geologic/geophysical characterization data. The test data are compared to previous numerical and experimental studies to determine how well this new experimental data compares with previous modeling, laboratory, and field tests. △ Less

Submitted 1 October, 2021; originally announced October 2021.

Report number: PNNL-SA-161540

Check It Again: Progressive Visual Question Answering via Visual Entailment

Authors: Qingyi Si, Zheng Lin, Mingyu Zheng, Peng Fu, Weiping Wang

Abstract: While sophisticated Visual Question Answering models have achieved remarkable success, they tend to answer questions only according to superficial correlations between question and answer. Several recent approaches have been developed to address this language priors problem. However, most of them predict the correct answer according to one best output without checking the authenticity of answers.… ▽ More While sophisticated Visual Question Answering models have achieved remarkable success, they tend to answer questions only according to superficial correlations between question and answer. Several recent approaches have been developed to address this language priors problem. However, most of them predict the correct answer according to one best output without checking the authenticity of answers. Besides, they only explore the interaction between image and question, ignoring the semantics of candidate answers. In this paper, we propose a select-and-rerank (SAR) progressive framework based on Visual Entailment. Specifically, we first select the candidate answers relevant to the question or the image, then we rerank the candidate answers by a visual entailment task, which verifies whether the image semantically entails the synthetic statement of the question and each candidate answer. Experimental results show the effectiveness of our proposed framework, which establishes a new state-of-the-art accuracy on VQA-CP v2 with a 7.55% improvement. △ Less

Submitted 8 June, 2021; originally announced June 2021.

Comments: ACL-2021