-
UAV's Rotor Micro-Doppler Feature Extraction Using Integrated Sensing and Communication Signal: Algorithm Design and Testbed Evaluation
Authors:
Jiachen Wei,
Dingyou Ma,
Feiyang He,
Qixun Zhang,
Zhiyong Feng,
Zhengfeng Liu,
Taohong Liang
Abstract:
With the rapid application of unmanned aerial vehicles (UAVs) in urban areas, the identification and tracking of hovering UAVs have become critical challenges, significantly impacting the safety of aircraft take-off and landing operations. As a promising technology for 6G mobile systems, integrated sensing and communication (ISAC) can be used to detect high-mobility UAVs with a low deployment cost…
▽ More
With the rapid application of unmanned aerial vehicles (UAVs) in urban areas, the identification and tracking of hovering UAVs have become critical challenges, significantly impacting the safety of aircraft take-off and landing operations. As a promising technology for 6G mobile systems, integrated sensing and communication (ISAC) can be used to detect high-mobility UAVs with a low deployment cost. The micro-Doppler signals from UAV rotors can be leveraged to address the detection of low-mobility and hovering UAVs using ISAC signals. However, determining whether the frame structure of the ISAC system can be used to identify UAVs, and how to accurately capture the weak rotor micro-Doppler signals of UAVs in complex environments, remain two challenging problems. This paper first proposes a novel frame structure for UAV micro-Doppler extraction and the representation of UAV micro-Doppler signals within the channel state information (CSI). Furthermore, to address complex environments and the interference caused by UAV body vibrations, the rotor micro-Doppler null space pursuit (rmD-NSP) algorithm and the feature extraction algorithm synchroextracting transform (SET) are designed to effectively separate UAV's rotor micro-Doppler signals and enhance their features in the spectrogram. Finally, both simulation and hardware testbed demonstrate that the proposed rmD-NSP algorithm enables the ISAC base station (BS) to accurately and completely extract UAV's rotor micro-Doppler signals. Within a 0.1s observation period, ISAC BS successfully captures eight rotations of the DJI M300 RTK UAV's rotor in urban environments. Compared to the existing AM-FM NSP and NSP signal decomposition algorithms, the integrity of the rotor micro-Doppler features is improved by 60%.
△ Less
Submitted 29 August, 2024;
originally announced August 2024.
-
Into the Unknown Unknowns: Engaged Human Learning through Participation in Language Model Agent Conversations
Authors:
Yucheng Jiang,
Yijia Shao,
Dekun Ma,
Sina J. Semnani,
Monica S. Lam
Abstract:
While language model (LM)-powered chatbots and generative search engines excel at answering concrete queries, discovering information in the terrain of unknown unknowns remains challenging for users. To emulate the common educational scenario where children/students learn by listening to and participating in conversations of their parents/teachers, we create Collaborative STORM (Co-STORM). Unlike…
▽ More
While language model (LM)-powered chatbots and generative search engines excel at answering concrete queries, discovering information in the terrain of unknown unknowns remains challenging for users. To emulate the common educational scenario where children/students learn by listening to and participating in conversations of their parents/teachers, we create Collaborative STORM (Co-STORM). Unlike QA systems that require users to ask all the questions, Co-STORM lets users observe and occasionally steer the discourse among several LM agents. The agents ask questions on the user's behalf, allowing the user to discover unknown unknowns serendipitously. To facilitate user interaction, Co-STORM assists users in tracking the discourse by organizing the uncovered information into a dynamic mind map, ultimately generating a comprehensive report as takeaways. For automatic evaluation, we construct the WildSeek dataset by collecting real information-seeking records with user goals. Co-STORM outperforms baseline methods on both discourse trace and report quality. In a further human evaluation, 70% of participants prefer Co-STORM over a search engine, and 78% favor it over a RAG chatbot.
△ Less
Submitted 27 August, 2024;
originally announced August 2024.
-
A Multiscale Gradient Fusion Method for Edge Detection in Color Images Utilizing the CBM3D Filter
Authors:
Zhuoyue Wang,
Yiyi Tao,
Danqing Ma
Abstract:
In this paper, a color edge detection strategy based on collaborative filtering combined with multiscale gradient fusion is proposed. The block-matching and 3D (BM3D) filter are used to enhance the sparse representation in the transform domain and achieve the effect of denoising, whereas the multiscale gradient fusion makes up for the defect of loss of details in single-scale edge detection and im…
▽ More
In this paper, a color edge detection strategy based on collaborative filtering combined with multiscale gradient fusion is proposed. The block-matching and 3D (BM3D) filter are used to enhance the sparse representation in the transform domain and achieve the effect of denoising, whereas the multiscale gradient fusion makes up for the defect of loss of details in single-scale edge detection and improves the edge detection resolution and quality. First, the RGB images in the dataset are converted to XYZ color space images through mathematical operations. Second, the colored block-matching and 3D (CBM3D) filter are used on the sparse images and to remove noise interference. Then, the vector gradients of the color image and the anisotropic Gaussian directional derivative of the two scale parameters are calculated and averaged pixel-by-pixel to obtain a new edge strength map. Finally, the edge features are enhanced by image normalization and non-maximum suppression technology, and on that basis, the edge contour is obtained by double threshold selection and a new morphological refinement method. Through an experimental analysis of the edge detection dataset, the method proposed has good noise robustness and high edge quality, which is better than the Color Sobel, Color Canny, SE and Color AGDD as shown by the PR curve, AUC, PSNR, MSE, and FOM indicators.
△ Less
Submitted 26 August, 2024;
originally announced August 2024.
-
Controllable Weyl Nodes and Fermi Arcs from Floquet Engineering Triple Fermions
Authors:
Shengpu Huang,
Fangyang Zhan,
Xianyong Ding,
Dong-Hui Xu,
Da-Shuai Ma,
Rui Wang
Abstract:
Floquet engineering with periodic driving as a powerful tool for designing desirable topological states has been the subject of intense recent studies. Here, we present the application of Floquet engineering to investigate evolution of topological triple fermions under irradiation of circularly polarized light (CPL), a phenomenon that currently remains a mystery. By using first-principles calculat…
▽ More
Floquet engineering with periodic driving as a powerful tool for designing desirable topological states has been the subject of intense recent studies. Here, we present the application of Floquet engineering to investigate evolution of topological triple fermions under irradiation of circularly polarized light (CPL), a phenomenon that currently remains a mystery. By using first-principles calculations and Floquet theorem, we demonstrate that WC-type TiO and its analogues are promising candidates for Floquet engineering of triple fermions. The symmetry analysis reveals that the electric field of CPL can break the specific symmetries, such as the time-reversal symmetry and its combination of spatial symmetries, inducing a transition to a flexibly controllable Weyl semimetallic phase. The survived spatial symmetry, controlled by light, guarantees that the Weyl nodes are located along the high-symmetry line or in high-symmetry planes in momentum space. Our findings focusing on Floquet engineering in realistic materials featured by triple fermions would facilitate both theoretical and experimental interest.
△ Less
Submitted 9 August, 2024;
originally announced August 2024.
-
Towards Real-Time Gaussian Splatting: Accelerating 3DGS through Photometric SLAM
Authors:
Yan Song Hu,
Dayou Mao,
Yuhao Chen,
John Zelek
Abstract:
Initial applications of 3D Gaussian Splatting (3DGS) in Visual Simultaneous Localization and Mapping (VSLAM) demonstrate the generation of high-quality volumetric reconstructions from monocular video streams. However, despite these promising advancements, current 3DGS integrations have reduced tracking performance and lower operating speeds compared to traditional VSLAM. To address these issues, w…
▽ More
Initial applications of 3D Gaussian Splatting (3DGS) in Visual Simultaneous Localization and Mapping (VSLAM) demonstrate the generation of high-quality volumetric reconstructions from monocular video streams. However, despite these promising advancements, current 3DGS integrations have reduced tracking performance and lower operating speeds compared to traditional VSLAM. To address these issues, we propose integrating 3DGS with Direct Sparse Odometry, a monocular photometric SLAM system. We have done preliminary experiments showing that using Direct Sparse Odometry point cloud outputs, as opposed to standard structure-from-motion methods, significantly shortens the training time needed to achieve high-quality renders. Reducing 3DGS training time enables the development of 3DGS-integrated SLAM systems that operate in real-time on mobile hardware. These promising initial findings suggest further exploration is warranted in combining traditional VSLAM systems with 3DGS.
△ Less
Submitted 7 August, 2024;
originally announced August 2024.
-
Hallu-PI: Evaluating Hallucination in Multi-modal Large Language Models within Perturbed Inputs
Authors:
Peng Ding,
Jingyu Wu,
Jun Kuang,
Dan Ma,
Xuezhi Cao,
Xunliang Cai,
Shi Chen,
Jiajun Chen,
Shujian Huang
Abstract:
Multi-modal Large Language Models (MLLMs) have demonstrated remarkable performance on various visual-language understanding and generation tasks. However, MLLMs occasionally generate content inconsistent with the given images, which is known as "hallucination". Prior works primarily center on evaluating hallucination using standard, unperturbed benchmarks, which overlook the prevalent occurrence o…
▽ More
Multi-modal Large Language Models (MLLMs) have demonstrated remarkable performance on various visual-language understanding and generation tasks. However, MLLMs occasionally generate content inconsistent with the given images, which is known as "hallucination". Prior works primarily center on evaluating hallucination using standard, unperturbed benchmarks, which overlook the prevalent occurrence of perturbed inputs in real-world scenarios-such as image cropping or blurring-that are critical for a comprehensive assessment of MLLMs' hallucination. In this paper, to bridge this gap, we propose Hallu-PI, the first benchmark designed to evaluate Hallucination in MLLMs within Perturbed Inputs. Specifically, Hallu-PI consists of seven perturbed scenarios, containing 1,260 perturbed images from 11 object types. Each image is accompanied by detailed annotations, which include fine-grained hallucination types, such as existence, attribute, and relation. We equip these annotations with a rich set of questions, making Hallu-PI suitable for both discriminative and generative tasks. Extensive experiments on 12 mainstream MLLMs, such as GPT-4V and Gemini-Pro Vision, demonstrate that these models exhibit significant hallucinations on Hallu-PI, which is not observed in unperturbed scenarios. Furthermore, our research reveals a severe bias in MLLMs' ability to handle different types of hallucinations. We also design two baselines specifically for perturbed scenarios, namely Perturbed-Reminder and Perturbed-ICL. We hope that our study will bring researchers' attention to the limitations of MLLMs when dealing with perturbed inputs, and spur further investigations to address this issue. Our code and datasets are publicly available at https://github.com/NJUNLP/Hallu-PI.
△ Less
Submitted 4 August, 2024; v1 submitted 2 August, 2024;
originally announced August 2024.
-
An Asynchronous Multi-core Accelerator for SNN inference
Authors:
Zhuo Chen,
De Ma,
Xiaofei Jin,
Qinghui Xing,
Ouwen Jin,
Xin Du,
Shuibing He,
Gang Pan
Abstract:
Spiking Neural Networks (SNNs) are extensively utilized in brain-inspired computing and neuroscience research. To enhance the speed and energy efficiency of SNNs, several many-core accelerators have been developed. However, maintaining the accuracy of SNNs often necessitates frequent explicit synchronization among all cores, which presents a challenge to overall efficiency. In this paper, we propo…
▽ More
Spiking Neural Networks (SNNs) are extensively utilized in brain-inspired computing and neuroscience research. To enhance the speed and energy efficiency of SNNs, several many-core accelerators have been developed. However, maintaining the accuracy of SNNs often necessitates frequent explicit synchronization among all cores, which presents a challenge to overall efficiency. In this paper, we propose an asynchronous architecture for Spiking Neural Networks (SNNs) that eliminates the need for inter-core synchronization, thus enhancing speed and energy efficiency. This approach leverages the pre-determined dependencies of neuromorphic cores established during compilation. Each core is equipped with a scheduler that monitors the status of its dependencies, allowing it to safely advance to the next timestep without waiting for other cores. This eliminates the necessity for global synchronization and minimizes core waiting time despite inherent workload imbalances. Comprehensive evaluations using five different SNN workloads show that our architecture achieves a 1.86x speedup and a 1.55x increase in energy efficiency compared to state-of-the-art synchronization architectures.
△ Less
Submitted 30 July, 2024;
originally announced July 2024.
-
Semantics Guided Disentangled GAN for Chest X-ray Image Rib Segmentation
Authors:
Lili Huang,
Dexin Ma,
Xiaowei Zhao,
Chenglong Li,
Haifeng Zhao,
Jin Tang,
Chuanfu Li
Abstract:
The label annotations for chest X-ray image rib segmentation are time consuming and laborious, and the labeling quality heavily relies on medical knowledge of annotators. To reduce the dependency on annotated data, existing works often utilize generative adversarial network (GAN) to generate training data. However, GAN-based methods overlook the nuanced information specific to individual organs, w…
▽ More
The label annotations for chest X-ray image rib segmentation are time consuming and laborious, and the labeling quality heavily relies on medical knowledge of annotators. To reduce the dependency on annotated data, existing works often utilize generative adversarial network (GAN) to generate training data. However, GAN-based methods overlook the nuanced information specific to individual organs, which degrades the generation quality of chest X-ray image. Hence, we propose a novel Semantics guided Disentangled GAN (SD-GAN), which can generate the high-quality training data by fully utilizing the semantic information of different organs, for chest X-ray image rib segmentation. In particular, we use three ResNet50 branches to disentangle features of different organs, then use a decoder to combine features and generate corresponding images. To ensure that the generated images correspond to the input organ labels in semantics tags, we employ a semantics guidance module to perform semantic guidance on the generated images. To evaluate the efficacy of SD-GAN in generating high-quality samples, we introduce modified TransUNet(MTUNet), a specialized segmentation network designed for multi-scale contextual information extracting and multi-branch decoding, effectively tackling the challenge of organ overlap. We also propose a new chest X-ray image dataset (CXRS). It includes 1250 samples from various medical institutions. Lungs, clavicles, and 24 ribs are simultaneously annotated on each chest X-ray image. The visualization and quantitative results demonstrate the efficacy of SD-GAN in generating high-quality chest X-ray image-mask pairs. Using generated data, our trained MTUNet overcomes the limitations of the data scale and outperforms other segmentation networks.
△ Less
Submitted 22 July, 2024;
originally announced July 2024.
-
avaTTAR: Table Tennis Stroke Training with On-body and Detached Visualization in Augmented Reality
Authors:
Dizhi Ma,
Xiyun Hu,
Jingyu Shi,
Mayank Patel,
Rahul Jain,
Ziyi Liu,
Zhengzhe Zhu,
Karthik Ramani
Abstract:
Table tennis stroke training is a critical aspect of player development. We designed a new augmented reality (AR) system, avaTTAR, for table tennis stroke training. The system provides both "on-body" (first-person view) and "detached" (third-person view) visual cues, enabling users to visualize target strokes and correct their attempts effectively with this dual perspectives setup. By employing a…
▽ More
Table tennis stroke training is a critical aspect of player development. We designed a new augmented reality (AR) system, avaTTAR, for table tennis stroke training. The system provides both "on-body" (first-person view) and "detached" (third-person view) visual cues, enabling users to visualize target strokes and correct their attempts effectively with this dual perspectives setup. By employing a combination of pose estimation algorithms and IMU sensors, avaTTAR captures and reconstructs the 3D body pose and paddle orientation of users during practice, allowing real-time comparison with expert strokes. Through a user study, we affirm avaTTAR's capacity to amplify player experience and training results.
△ Less
Submitted 26 July, 2024; v1 submitted 22 July, 2024;
originally announced July 2024.
-
Skew-scattering Pockels effect and metallic electro-optics in gapped bilayer graphene
Authors:
Da Ma,
Ying Xiong,
Justin C. W. Song
Abstract:
We argue that a range of strong metallic electro-optic (EO) effects can be naturally realized from non-Drude dynamics of free carriers in metals. In particular, in clean metals we identify skew-scattering and a "Snap" (third-order derivative of velocity) dominating the Pockels and Kerr EO behavior of metals in the clean limit. Strikingly, we find that both Pockels and Kerr EO in metals play critic…
▽ More
We argue that a range of strong metallic electro-optic (EO) effects can be naturally realized from non-Drude dynamics of free carriers in metals. In particular, in clean metals we identify skew-scattering and a "Snap" (third-order derivative of velocity) dominating the Pockels and Kerr EO behavior of metals in the clean limit. Strikingly, we find that both Pockels and Kerr EO in metals play critical roles in metallic EO phenomena: for instance, metallic Pockels and Kerr EO effectively compete to produce a field-activated birefringence that is non-reciprocal in applied DC fields. Similarly, both contribute to sizeable field-induced modulations to transmission and reflection across a range of frequencies. We find metallic EO effects can be naturally realized in layered 2D materials such as gapped bilayer graphene producing pronounced values of EO coefficients in the terahertz -- an interesting new metallic platform for terahertz electro-optic modulation.
△ Less
Submitted 16 July, 2024;
originally announced July 2024.
-
VLMPC: Vision-Language Model Predictive Control for Robotic Manipulation
Authors:
Wentao Zhao,
Jiaming Chen,
Ziyu Meng,
Donghui Mao,
Ran Song,
Wei Zhang
Abstract:
Although Model Predictive Control (MPC) can effectively predict the future states of a system and thus is widely used in robotic manipulation tasks, it does not have the capability of environmental perception, leading to the failure in some complex scenarios. To address this issue, we introduce Vision-Language Model Predictive Control (VLMPC), a robotic manipulation framework which takes advantage…
▽ More
Although Model Predictive Control (MPC) can effectively predict the future states of a system and thus is widely used in robotic manipulation tasks, it does not have the capability of environmental perception, leading to the failure in some complex scenarios. To address this issue, we introduce Vision-Language Model Predictive Control (VLMPC), a robotic manipulation framework which takes advantage of the powerful perception capability of vision language model (VLM) and integrates it with MPC. Specifically, we propose a conditional action sampling module which takes as input a goal image or a language instruction and leverages VLM to sample a set of candidate action sequences. Then, a lightweight action-conditioned video prediction model is designed to generate a set of future frames conditioned on the candidate action sequences. VLMPC produces the optimal action sequence with the assistance of VLM through a hierarchical cost function that formulates both pixel-level and knowledge-level consistence between the current observation and the goal image. We demonstrate that VLMPC outperforms the state-of-the-art methods on public benchmarks. More importantly, our method showcases excellent performance in various real-world tasks of robotic manipulation. Code is available at~\url{https://github.com/PPjmchen/VLMPC}.
△ Less
Submitted 13 July, 2024;
originally announced July 2024.
-
RespEar: Earable-Based Robust Respiratory Rate Monitoring
Authors:
Yang Liu,
Kayla-Jade Butkow,
Jake Stuchbury-Wass,
Adam Pullin,
Dong Ma,
Cecilia Mascolo
Abstract:
Respiratory rate (RR) monitoring is integral to understanding physical and mental health and tracking fitness. Existing studies have demonstrated the feasibility of RR monitoring under specific user conditions (e.g., while remaining still, or while breathing heavily). Yet, performing accurate, continuous and non-obtrusive RR monitoring across diverse daily routines and activities remains challengi…
▽ More
Respiratory rate (RR) monitoring is integral to understanding physical and mental health and tracking fitness. Existing studies have demonstrated the feasibility of RR monitoring under specific user conditions (e.g., while remaining still, or while breathing heavily). Yet, performing accurate, continuous and non-obtrusive RR monitoring across diverse daily routines and activities remains challenging. In this work, we present RespEar, an earable-based system for robust RR monitoring. By leveraging the unique properties of in-ear microphones in earbuds, RespEar enables the use of Respiratory Sinus Arrhythmia (RSA) and Locomotor Respiratory Coupling (LRC), physiological couplings between cardiovascular activity, gait and respiration, to indirectly determine RR. This effectively addresses the challenges posed by the almost imperceptible breathing signals under daily activities. We further propose a suite of meticulously crafted signal processing schemes to improve RR estimation accuracy and robustness. With data collected from 18 subjects over 8 activities, RespEar measures RR with a mean absolute error (MAE) of 1.48 breaths per minutes (BPM) and a mean absolute percent error (MAPE) of 9.12% in sedentary conditions, and a MAE of 2.28 BPM and a MAPE of 11.04% in active conditions, respectively, which is unprecedented for a method capable of generalizing across conditions with a single modality.
△ Less
Submitted 9 July, 2024;
originally announced July 2024.
-
Interference Management in MIMO-ISAC Systems: A Transceiver Design Approach
Authors:
Yangyang Niu,
Zhiqing Wei,
Dingyou Ma,
Xiaoyu Yang,
Huici Wu,
Zhiyong Feng,
Jianhua Yuan
Abstract:
The integrated sensing and communication (ISAC) system under multi-input multi-output (MIMO) architecture achieves dual functionalities of sensing and communication on the same platform by utilizing spatial gain, which provides a feasible paradigm facing spectrum congestion. However, the dual functionalities of sensing and communication operating simultaneously in the same platform bring severe in…
▽ More
The integrated sensing and communication (ISAC) system under multi-input multi-output (MIMO) architecture achieves dual functionalities of sensing and communication on the same platform by utilizing spatial gain, which provides a feasible paradigm facing spectrum congestion. However, the dual functionalities of sensing and communication operating simultaneously in the same platform bring severe interference in the ISAC systems. Facing this challenge, we propose a joint optimization framework for transmit beamforming and receive filter design for ISAC systems with MIMO architecture. We aim to maximize the signal-to-clutter-plus-noise ratio (SCNR) at the receiver while considering various constraints such as waveform similarity, power budget, and communication performance requirements to ensure the integration of the dual functionalities. In particular, the overall transmit beamforming is refined into sensing beamforming and communication beamforming, and a quadratic transformation (QT) is introduced to relax and convert the complex non-convex optimization objective. An efficient algorithm based on covariance matrix tapers (CMT) is proposed to restructure the clutter covariance matrix considering the mismatched steering vector, thereby improving the robustness of the ISAC transceiver design. Numerical simulations are provided to demonstrate the effectiveness of the proposed algorithm.
△ Less
Submitted 7 July, 2024;
originally announced July 2024.
-
CLIMB: A Benchmark of Clinical Bias in Large Language Models
Authors:
Yubo Zhang,
Shudi Hou,
Mingyu Derek Ma,
Wei Wang,
Muhao Chen,
Jieyu Zhao
Abstract:
Large language models (LLMs) are increasingly applied to clinical decision-making. However, their potential to exhibit bias poses significant risks to clinical equity. Currently, there is a lack of benchmarks that systematically evaluate such clinical bias in LLMs. While in downstream tasks, some biases of LLMs can be avoided such as by instructing the model to answer "I'm not sure...", the intern…
▽ More
Large language models (LLMs) are increasingly applied to clinical decision-making. However, their potential to exhibit bias poses significant risks to clinical equity. Currently, there is a lack of benchmarks that systematically evaluate such clinical bias in LLMs. While in downstream tasks, some biases of LLMs can be avoided such as by instructing the model to answer "I'm not sure...", the internal bias hidden within the model still lacks deep studies. We introduce CLIMB (shorthand for A Benchmark of Clinical Bias in Large Language Models), a pioneering comprehensive benchmark to evaluate both intrinsic (within LLMs) and extrinsic (on downstream tasks) bias in LLMs for clinical decision tasks. Notably, for intrinsic bias, we introduce a novel metric, AssocMAD, to assess the disparities of LLMs across multiple demographic groups. Additionally, we leverage counterfactual intervention to evaluate extrinsic bias in a task of clinical diagnosis prediction. Our experiments across popular and medically adapted LLMs, particularly from the Mistral and LLaMA families, unveil prevalent behaviors with both intrinsic and extrinsic bias. This work underscores the critical need to mitigate clinical bias and sets a new standard for future evaluations of LLMs' clinical bias.
△ Less
Submitted 6 July, 2024;
originally announced July 2024.
-
Color-map recommendation for MR relaxometry maps
Authors:
Miha Fuderer,
Barbara Wichtmann,
Fabio Crameri,
Nandita M. deSouza,
Bettina Baeßler,
Vikas Gulani,
Meiyun Wang,
Dirk Poot,
Ruud de Boer,
Matt Cashmore,
Wolter de Graaf,
Kathryn E. Keenan,
Dan Ma,
Carolin Pirkl,
Nico Sollmann,
Sebastian Weingärtner,
Stefano Mandija,
Xavier Golay
Abstract:
Purpose: To harmonize the use of color for MR relaxometry maps and therefore recommend the use of specific color-maps for representing T1 and T2 maps. Methods: Perceptually linearized color-maps were chosen to have similar color settings as those proposed by Griswold et al. in 2018. A Delphi process, polling the opinion of a panel of 81 experts, was used to generate consensus on the suitability of…
▽ More
Purpose: To harmonize the use of color for MR relaxometry maps and therefore recommend the use of specific color-maps for representing T1 and T2 maps. Methods: Perceptually linearized color-maps were chosen to have similar color settings as those proposed by Griswold et al. in 2018. A Delphi process, polling the opinion of a panel of 81 experts, was used to generate consensus on the suitability of these maps. Results: Consensus was reached on the suitability of the logarithm-processed Lipari color-map for T1 and the logarithm-processed Navia color-map for T2. There was consensus on color bars being mandatory and on the use of a specific value indicating invalidity. There was no consensus on whether the ranges should be fixed per anatomy. Conclusion: The authors recommend the use of the logarithm-processed Lipari color map for displaying quantitative T1 maps and R1 maps; likewise, the authors recommend the logarithm-processed Navia color-map for displaying T2, T2*, R2 and R2* maps.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
Adaptive sampling strategy for tolerance analysis of freeform optical surfaces based on critical ray aiming
Authors:
Rundong Fan,
Shili Wei,
Zhuang Qian,
Huiru Ji,
Hao Tan,
Yan Mo,
Donglin Ma
Abstract:
The tolerance analysis of freeform surfaces plays a crucial role in the development of advanced imaging systems. However, the intricate relationship between surface error and imaging quality poses significant challenges, necessitating dense sampling of featured rays during the computation process to ensure an accurate tolerance for different fields of view (FOVs). Here, we propose an adaptive samp…
▽ More
The tolerance analysis of freeform surfaces plays a crucial role in the development of advanced imaging systems. However, the intricate relationship between surface error and imaging quality poses significant challenges, necessitating dense sampling of featured rays during the computation process to ensure an accurate tolerance for different fields of view (FOVs). Here, we propose an adaptive sampling strategy called "Critical Ray Aiming" for surface tolerance analysis. By identifying the most sensitive ray to wave aberration at each surface point, our methodology facilitates flexible sampling of the FOVs and entrance pupil (EP), achieving computational efficiency without compromising accuracy in determining tolerable surface error. We demonstrate the effectiveness of our method through tolerance analysis of two different freeform imaging systems.
△ Less
Submitted 4 July, 2024;
originally announced July 2024.
-
MIRAI: Evaluating LLM Agents for Event Forecasting
Authors:
Chenchen Ye,
Ziniu Hu,
Yihe Deng,
Zijie Huang,
Mingyu Derek Ma,
Yanqiao Zhu,
Wei Wang
Abstract:
Recent advancements in Large Language Models (LLMs) have empowered LLM agents to autonomously collect world information, over which to conduct reasoning to solve complex problems. Given this capability, increasing interests have been put into employing LLM agents for predicting international events, which can influence decision-making and shape policy development on an international scale. Despite…
▽ More
Recent advancements in Large Language Models (LLMs) have empowered LLM agents to autonomously collect world information, over which to conduct reasoning to solve complex problems. Given this capability, increasing interests have been put into employing LLM agents for predicting international events, which can influence decision-making and shape policy development on an international scale. Despite such a growing interest, there is a lack of a rigorous benchmark of LLM agents' forecasting capability and reliability. To address this gap, we introduce MIRAI, a novel benchmark designed to systematically evaluate LLM agents as temporal forecasters in the context of international events. Our benchmark features an agentic environment with tools for accessing an extensive database of historical, structured events and textual news articles. We refine the GDELT event database with careful cleaning and parsing to curate a series of relational prediction tasks with varying forecasting horizons, assessing LLM agents' abilities from short-term to long-term forecasting. We further implement APIs to enable LLM agents to utilize different tools via a code-based interface. In summary, MIRAI comprehensively evaluates the agents' capabilities in three dimensions: 1) autonomously source and integrate critical information from large global databases; 2) write codes using domain-specific APIs and libraries for tool-use; and 3) jointly reason over historical knowledge from diverse formats and time to accurately predict future events. Through comprehensive benchmarking, we aim to establish a reliable framework for assessing the capabilities of LLM agents in forecasting international events, thereby contributing to the development of more accurate and trustworthy models for international relation analysis.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
-
Correlation entropy of free semigroup actions
Authors:
Xiaojiang Ye,
Yanjie Tang,
Dongkui Ma
Abstract:
This paper introduces the concepts of correlation entropy and local correlation entropy for free semigroup actions on compact metric space, and explores their fundamental properties. Thereafter, we generalize some classical results on correlation entropy and local correlation entropy to apply to free semigroup actions. Finally, we establish the relationship between topological entropy, measure-the…
▽ More
This paper introduces the concepts of correlation entropy and local correlation entropy for free semigroup actions on compact metric space, and explores their fundamental properties. Thereafter, we generalize some classical results on correlation entropy and local correlation entropy to apply to free semigroup actions. Finally, we establish the relationship between topological entropy, measure-theoretic entropy, correlation entropy, and local correlation entropy for free semigroup actions under various conditions.
△ Less
Submitted 26 June, 2024;
originally announced June 2024.
-
Realizing a spatially correlated lattice interferometer
Authors:
Peng Peng,
Dekai Mao,
Yi Liang,
Guoling Yin,
Hongmian Shui,
Bo Song,
Xiaoji Zhou
Abstract:
Atom interferometers provide a powerful tool for measuring physical constants and testifying fundamental physics with unprecedented precision. Conventional atom interferometry focuses on the phase difference between two paths and utilizes matter waves with fixed coherence. Here, we report on realizing a Ramsey-Bordé interferometer of coherent matter waves dressed by a moving optical lattice in the…
▽ More
Atom interferometers provide a powerful tool for measuring physical constants and testifying fundamental physics with unprecedented precision. Conventional atom interferometry focuses on the phase difference between two paths and utilizes matter waves with fixed coherence. Here, we report on realizing a Ramsey-Bordé interferometer of coherent matter waves dressed by a moving optical lattice in the gravity direction, and explore the resulting interference along multiple paths with tunable coherence. We investigate spatial correlations of atoms both within the lattice and between two arms by interferometry, and observe the emerging multiple interference peaks owing to the long-range coherence nature of the Bose-Einstein condensate. Our findings agree well with theoretical simulations, paving the way for high-precision interferometry with ultracold atoms.
△ Less
Submitted 24 June, 2024;
originally announced June 2024.
-
Demonstration of High-Efficiency Microwave Heating Producing Record Highly Charged Xenon Ion Beams with Superconducting ECR Ion Sources
Authors:
X. Wang,
J. B. Li,
V. Mironov,
J. W. Guo,
X. Z. Zhang,
O. Tarvainen,
Y. C. Feng,
L. X. Li,
J. D. Ma,
Z. H. Zhang,
W. Lu,
S. Bogomolov,
L. Sun,
H. W. Zhao
Abstract:
Intense highly charged ion beam production is essential for high-power heavy ion accelerators. A novel movable Vlasov launcher for superconducting high charge state Electron Cyclotron Resonance (ECR) ion source has been devised that can affect the microwave power effectiveness by a factor of about 4 in terms of highly charged ion beam production. This approach based on a dedicated microwave launch…
▽ More
Intense highly charged ion beam production is essential for high-power heavy ion accelerators. A novel movable Vlasov launcher for superconducting high charge state Electron Cyclotron Resonance (ECR) ion source has been devised that can affect the microwave power effectiveness by a factor of about 4 in terms of highly charged ion beam production. This approach based on a dedicated microwave launching system instead of the traditional coupling scheme has led to new insight on microwave-plasma interaction. With this new understanding, the world record highly charged xenon ion beam currents have been enhanced by up to a factor of 2, which could directly and significantly enhance the performance of heavy ion accelerators and provide many new research opportunities in nuclear physics, atomic physics and other disciplines.
△ Less
Submitted 14 July, 2024; v1 submitted 19 June, 2024;
originally announced June 2024.
-
Hybrid Beamforming Design for Near-Field ISAC with Modular XL-MIMO
Authors:
Chunwei Meng,
Dingyou Ma,
Zhaolin Wang,
Yuanwei Liu,
Zhiqing Wei,
Zhiyong Feng
Abstract:
A novel modular extremely large-scale multiple-input-multiple-output (XL-MIMO) integrated sensing and communication (ISAC) framework is proposed in this paper. We consider a downlink ISAC scenario and exploit the modular array architecture to enhance the communication spectral efficiency and sensing resolution while reducing the channel modeling complexity by employing the hybrid spherical and pla…
▽ More
A novel modular extremely large-scale multiple-input-multiple-output (XL-MIMO) integrated sensing and communication (ISAC) framework is proposed in this paper. We consider a downlink ISAC scenario and exploit the modular array architecture to enhance the communication spectral efficiency and sensing resolution while reducing the channel modeling complexity by employing the hybrid spherical and planar wavefront model. Considering the hybrid digital-analog structure inherent to modular arrays, we formulate a joint analog-digital beamforming design problem based on the communication spectral efficiency and sensing signal-to-clutter-plus-noise ratio (SCNR). By exploring the structural similarity of the communication and sensing channels, it is proved that the optimal transmit covariance matrix lies in the subspace spanned by the subarray response vectors, yielding a closed-form solution for the optimal analog beamformer. Consequently, the joint design problem is transformed into a low-dimensional rank-constrained digital beamformer optimization. We first propose a manifold optimization method that directly optimizes the digital beamformer on the rank-constrained Stiefel manifold. Additionally, we develop an semidefinite relaxation (SDR)-based approach that relaxes the rank constraint and employ the randomization technique to obtain a near-optimal solution. Simulation results demonstrate the effectiveness of the proposed modular XL-MIMO ISAC framework and algorithms.
△ Less
Submitted 18 June, 2024;
originally announced June 2024.
-
VideoLLM-online: Online Video Large Language Model for Streaming Video
Authors:
Joya Chen,
Zhaoyang Lv,
Shiwei Wu,
Kevin Qinghong Lin,
Chenan Song,
Difei Gao,
Jia-Wei Liu,
Ziteng Gao,
Dongxing Mao,
Mike Zheng Shou
Abstract:
Recent Large Language Models have been enhanced with vision capabilities, enabling them to comprehend images, videos, and interleaved vision-language content. However, the learning methods of these large multimodal models typically treat videos as predetermined clips, making them less effective and efficient at handling streaming video inputs. In this paper, we propose a novel Learning-In-Video-St…
▽ More
Recent Large Language Models have been enhanced with vision capabilities, enabling them to comprehend images, videos, and interleaved vision-language content. However, the learning methods of these large multimodal models typically treat videos as predetermined clips, making them less effective and efficient at handling streaming video inputs. In this paper, we propose a novel Learning-In-Video-Stream (LIVE) framework, which enables temporally aligned, long-context, and real-time conversation within a continuous video stream. Our LIVE framework comprises comprehensive approaches to achieve video streaming dialogue, encompassing: (1) a training objective designed to perform language modeling for continuous streaming inputs, (2) a data generation scheme that converts offline temporal annotations into a streaming dialogue format, and (3) an optimized inference pipeline to speed up the model responses in real-world video streams. With our LIVE framework, we built VideoLLM-online model upon Llama-2/Llama-3 and demonstrate its significant advantages in processing streaming videos. For instance, on average, our model can support streaming dialogue in a 5-minute video clip at over 10 FPS on an A100 GPU. Moreover, it also showcases state-of-the-art performance on public offline video benchmarks, such as recognition, captioning, and forecasting. The code, model, data, and demo have been made available at https://showlab.github.io/videollm-online.
△ Less
Submitted 17 June, 2024;
originally announced June 2024.
-
CliBench: Multifaceted Evaluation of Large Language Models in Clinical Decisions on Diagnoses, Procedures, Lab Tests Orders and Prescriptions
Authors:
Mingyu Derek Ma,
Chenchen Ye,
Yu Yan,
Xiaoxuan Wang,
Peipei Ping,
Timothy S Chang,
Wei Wang
Abstract:
The integration of Artificial Intelligence (AI), especially Large Language Models (LLMs), into the clinical diagnosis process offers significant potential to improve the efficiency and accessibility of medical care. While LLMs have shown some promise in the medical domain, their application in clinical diagnosis remains underexplored, especially in real-world clinical practice, where highly sophis…
▽ More
The integration of Artificial Intelligence (AI), especially Large Language Models (LLMs), into the clinical diagnosis process offers significant potential to improve the efficiency and accessibility of medical care. While LLMs have shown some promise in the medical domain, their application in clinical diagnosis remains underexplored, especially in real-world clinical practice, where highly sophisticated, patient-specific decisions need to be made. Current evaluations of LLMs in this field are often narrow in scope, focusing on specific diseases or specialties and employing simplified diagnostic tasks. To bridge this gap, we introduce CliBench, a novel benchmark developed from the MIMIC IV dataset, offering a comprehensive and realistic assessment of LLMs' capabilities in clinical diagnosis. This benchmark not only covers diagnoses from a diverse range of medical cases across various specialties but also incorporates tasks of clinical significance: treatment procedure identification, lab test ordering and medication prescriptions. Supported by structured output ontologies, CliBench enables a precise and multi-granular evaluation, offering an in-depth understanding of LLM's capability on diverse clinical tasks of desired granularity. We conduct a zero-shot evaluation of leading LLMs to assess their proficiency in clinical decision-making. Our preliminary results shed light on the potential and limitations of current LLMs in clinical settings, providing valuable insights for future advancements in LLM-powered healthcare.
△ Less
Submitted 14 June, 2024;
originally announced June 2024.
-
MuirBench: A Comprehensive Benchmark for Robust Multi-image Understanding
Authors:
Fei Wang,
Xingyu Fu,
James Y. Huang,
Zekun Li,
Qin Liu,
Xiaogeng Liu,
Mingyu Derek Ma,
Nan Xu,
Wenxuan Zhou,
Kai Zhang,
Tianyi Lorena Yan,
Wenjie Jacky Mo,
Hsiang-Hui Liu,
Pan Lu,
Chunyuan Li,
Chaowei Xiao,
Kai-Wei Chang,
Dan Roth,
Sheng Zhang,
Hoifung Poon,
Muhao Chen
Abstract:
We introduce MuirBench, a comprehensive benchmark that focuses on robust multi-image understanding capabilities of multimodal LLMs. MuirBench consists of 12 diverse multi-image tasks (e.g., scene understanding, ordering) that involve 10 categories of multi-image relations (e.g., multiview, temporal relations). Comprising 11,264 images and 2,600 multiple-choice questions, MuirBench is created in a…
▽ More
We introduce MuirBench, a comprehensive benchmark that focuses on robust multi-image understanding capabilities of multimodal LLMs. MuirBench consists of 12 diverse multi-image tasks (e.g., scene understanding, ordering) that involve 10 categories of multi-image relations (e.g., multiview, temporal relations). Comprising 11,264 images and 2,600 multiple-choice questions, MuirBench is created in a pairwise manner, where each standard instance is paired with an unanswerable variant that has minimal semantic differences, in order for a reliable assessment. Evaluated upon 20 recent multi-modal LLMs, our results reveal that even the best-performing models like GPT-4o and Gemini Pro find it challenging to solve MuirBench, achieving 68.0% and 49.3% in accuracy. Open-source multimodal LLMs trained on single images can hardly generalize to multi-image questions, hovering below 33.3% in accuracy. These results highlight the importance of MuirBench in encouraging the community to develop multimodal LLMs that can look beyond a single image, suggesting potential pathways for future improvements.
△ Less
Submitted 1 July, 2024; v1 submitted 13 June, 2024;
originally announced June 2024.
-
Cohomology of a restricted Lie algebra with a restricted derivation in characteristic 2
Authors:
Dan Mao,
Liangyun Chen
Abstract:
This paper mainly studies the ResLieDer pair in characteristic 2, that is, a restricted Lie algebra with a restricted derivation. We define the restricted representation of a ResLieDer pair and the corresponding cohomology complex. We show that a ResLieDer pair is rigid if the second cohomology group is trivial and a deformation of order $n$ is extensible if and only if its obstruction class is tr…
▽ More
This paper mainly studies the ResLieDer pair in characteristic 2, that is, a restricted Lie algebra with a restricted derivation. We define the restricted representation of a ResLieDer pair and the corresponding cohomology complex. We show that a ResLieDer pair is rigid if the second cohomology group is trivial and a deformation of order $n$ is extensible if and only if its obstruction class is trivial. Moreover, we prove that the central extensions of a ResLieDer pair are classified by the second cohomology group. Finally, we show that a pair of restricted derivations is extensible if and only if its obstruction class is trivial.
△ Less
Submitted 12 February, 2024;
originally announced June 2024.
-
Evolving Subnetwork Training for Large Language Models
Authors:
Hanqi Li,
Lu Chen,
Da Ma,
Zijian Wu,
Su Zhu,
Kai Yu
Abstract:
Large language models have ushered in a new era of artificial intelligence research. However, their substantial training costs hinder further development and widespread adoption. In this paper, inspired by the redundancy in the parameters of large language models, we propose a novel training paradigm: Evolving Subnetwork Training (EST). EST samples subnetworks from the layers of the large language…
▽ More
Large language models have ushered in a new era of artificial intelligence research. However, their substantial training costs hinder further development and widespread adoption. In this paper, inspired by the redundancy in the parameters of large language models, we propose a novel training paradigm: Evolving Subnetwork Training (EST). EST samples subnetworks from the layers of the large language model and from commonly used modules within each layer, Multi-Head Attention (MHA) and Multi-Layer Perceptron (MLP). By gradually increasing the size of the subnetworks during the training process, EST can save the cost of training. We apply EST to train GPT2 model and TinyLlama model, resulting in 26.7\% FLOPs saving for GPT2 and 25.0\% for TinyLlama without an increase in loss on the pre-training dataset. Moreover, EST leads to performance improvements in downstream tasks, indicating that it benefits generalization. Additionally, we provide intuitive theoretical studies based on training dynamics and Dropout theory to ensure the feasibility of EST. Our code is available at https://github.com/OpenDFM/EST.
△ Less
Submitted 11 June, 2024;
originally announced June 2024.
-
Classification of Fermi Gamma-Ray Bursts Based on Machine Learning
Authors:
Si-Yuan Zhu,
Wan-Peng Sun,
Da-Ling Ma,
Fu-Wen Zhang
Abstract:
Gamma-ray bursts (GRBs) are typically classified into long and short GRBs based on their durations. However, there is a significant overlapping in the duration distributions of these two categories. In this paper, we apply the unsupervised dimensionality reduction algorithm called t-SNE and UMAP to classify 2061 Fermi GRBs based on four observed quantities: duration, peak energy, fluence, and peak…
▽ More
Gamma-ray bursts (GRBs) are typically classified into long and short GRBs based on their durations. However, there is a significant overlapping in the duration distributions of these two categories. In this paper, we apply the unsupervised dimensionality reduction algorithm called t-SNE and UMAP to classify 2061 Fermi GRBs based on four observed quantities: duration, peak energy, fluence, and peak flux. The map results of t-SNE and UMAP show a clear division of these GRBs into two clusters. We mark the two clusters as GRBs-I and GRBs-II, and find that all GRBs associated with supernovae are classified as GRBs-II. It includes the peculiar short GRB 200826A, which was confirmed to originate from the death of a massive star. Furthermore, except for two extreme events GRB 211211A and GRB 230307A, all GRBs associated with kilonovae fall into GRBs-I population. By comparing to the traditional classification of short and long GRBs, the distribution of durations for GRBs-I and GRBs-II do not have a fixed boundary. We find that more than 10% of GRBs-I have a duration greater than 2 seconds, while approximately 1% of GRBs-II have a duration shorter than 2 seconds.
△ Less
Submitted 8 June, 2024;
originally announced June 2024.
-
Sparsity-Accelerated Training for Large Language Models
Authors:
Da Ma,
Lu Chen,
Pengyu Wang,
Hongshen Xu,
Hanqi Li,
Liangtai Sun,
Su Zhu,
Shuai Fan,
Kai Yu
Abstract:
Large language models (LLMs) have demonstrated proficiency across various natural language processing (NLP) tasks but often require additional training, such as continual pre-training and supervised fine-tuning. However, the costs associated with this, primarily due to their large parameter count, remain high. This paper proposes leveraging \emph{sparsity} in pre-trained LLMs to expedite this trai…
▽ More
Large language models (LLMs) have demonstrated proficiency across various natural language processing (NLP) tasks but often require additional training, such as continual pre-training and supervised fine-tuning. However, the costs associated with this, primarily due to their large parameter count, remain high. This paper proposes leveraging \emph{sparsity} in pre-trained LLMs to expedite this training process. By observing sparsity in activated neurons during forward iterations, we identify the potential for computational speed-ups by excluding inactive neurons. We address associated challenges by extending existing neuron importance evaluation metrics and introducing a ladder omission rate scheduler. Our experiments on Llama-2 demonstrate that Sparsity-Accelerated Training (SAT) achieves comparable or superior performance to standard training while significantly accelerating the process. Specifically, SAT achieves a $45\%$ throughput improvement in continual pre-training and saves $38\%$ training time in supervised fine-tuning in practice. It offers a simple, hardware-agnostic, and easily deployable framework for additional LLM training. Our code is available at https://github.com/OpenDFM/SAT.
△ Less
Submitted 6 June, 2024; v1 submitted 3 June, 2024;
originally announced June 2024.
-
Accurate Patient Alignment without Unnecessary Imaging Dose via Synthesizing Patient-specific 3D CT Images from 2D kV Images
Authors:
Yuzhen Ding,
Jason M. Holmes,
Hongying Feng,
Baoxin Li,
Lisa A. McGee,
Jean-Claude M. Rwigema,
Sujay A. Vora,
Daniel J. Ma,
Robert L. Foote,
Samir H. Patel,
Wei Liu
Abstract:
In radiotherapy, 2D orthogonally projected kV images are used for patient alignment when 3D-on-board imaging(OBI) unavailable. But tumor visibility is constrained due to the projection of patient's anatomy onto a 2D plane, potentially leading to substantial setup errors. In treatment room with 3D-OBI such as cone beam CT(CBCT), the field of view(FOV) of CBCT is limited with unnecessarily high imag…
▽ More
In radiotherapy, 2D orthogonally projected kV images are used for patient alignment when 3D-on-board imaging(OBI) unavailable. But tumor visibility is constrained due to the projection of patient's anatomy onto a 2D plane, potentially leading to substantial setup errors. In treatment room with 3D-OBI such as cone beam CT(CBCT), the field of view(FOV) of CBCT is limited with unnecessarily high imaging dose, thus unfavorable for pediatric patients. A solution to this dilemma is to reconstruct 3D CT from kV images obtained at the treatment position. Here, we propose a dual-models framework built with hierarchical ViT blocks. Unlike a proof-of-concept approach, our framework considers kV images as the solo input and can synthesize accurate, full-size 3D CT in real time(within milliseconds). We demonstrate the feasibility of the proposed approach on 10 patients with head and neck (H&N) cancer using image quality(MAE: <45HU), dosimetrical accuracy(Gamma passing rate (2%/2mm/10%)>97%) and patient position uncertainty(shift error: <0.4mm). The proposed framework can generate accurate 3D CT faithfully mirroring real-time patient position, thus significantly improving patient setup accuracy, keeping imaging dose minimum, and maintaining treatment veracity.
△ Less
Submitted 1 April, 2024;
originally announced May 2024.
-
Atomic transport dynamics in crossed optical dipole trap
Authors:
Peng Peng,
Zhengxi Zhang,
Yaoyuan Fan,
Guoling Yin,
Dekai Mao,
Xuzong Chen,
Wei Xiong,
Xiaoji Zhou
Abstract:
We study the dynamical evolution of cold atoms in crossed optical dipole trap theoretically and experimentally. The atomic transport process is accompanied by two competitive kinds of physical mechanics, atomic loading and atomic loss. The loading process normally is negligible in the evaporative cooling experiment on the ground, while it is significant in the preparation of ultra-cold atoms in th…
▽ More
We study the dynamical evolution of cold atoms in crossed optical dipole trap theoretically and experimentally. The atomic transport process is accompanied by two competitive kinds of physical mechanics, atomic loading and atomic loss. The loading process normally is negligible in the evaporative cooling experiment on the ground, while it is significant in the preparation of ultra-cold atoms in the space station. Normally, the atomic loading process is much weaker than the atomic loss process, and the atomic number in the center region of the trap decreases monotonically, as reported in previous research. However, when the atomic loading process is comparable to the atomic loss process, the atomic number in the center region of the trap will initially increase to a maximum value and then slowly decrease, and we have observed the phenomenon first. The increase of atomic number in the center region of the trap shows the presence of the loading process, and this will be significant especially under microgravity conditions. We build a theoretical model to analyze the competitive relationship, which coincides with the experimental results well. Furthermore, we have also given the predicted evolutionary behaviors under different conditions. This research provides a solid foundation for further understanding of the atomic transport process in traps. The analysis of loading process is of significant importance for the preparation of ultra-cold atoms in a crossed optical dipole trap under microgravity conditions.
△ Less
Submitted 15 May, 2024;
originally announced May 2024.
-
Multi-Objective Optimization-based Transmit Beamforming for Multi-Target and Multi-User MIMO-ISAC Systems
Authors:
Chunwei Meng,
Zhiqing Wei,
Dingyou Ma,
Wanli Ni,
Liyan Su,
Zhiyong Feng
Abstract:
Integrated sensing and communication (ISAC) is an enabling technology for the sixth-generation mobile communications, which equips the wireless communication networks with sensing capabilities. In this paper, we investigate transmit beamforming design for multiple-input and multiple-output (MIMO)-ISAC systems in scenarios with multiple radar targets and communication users. A general form of multi…
▽ More
Integrated sensing and communication (ISAC) is an enabling technology for the sixth-generation mobile communications, which equips the wireless communication networks with sensing capabilities. In this paper, we investigate transmit beamforming design for multiple-input and multiple-output (MIMO)-ISAC systems in scenarios with multiple radar targets and communication users. A general form of multi-target sensing mutual information (MI) is derived, along with its upper bound, which can be interpreted as the sum of individual single-target sensing MI. Additionally, this upper bound can be achieved by suppressing the cross-correlation among reflected signals from different targets, which aligns with the principles of adaptive MIMO radar. Then, we propose a multi-objective optimization framework based on the signal-to-interference-plus-noise ratio of each user and the tight upper bound of sensing MI, introducing the Pareto boundary to characterize the achievable communication-sensing performance boundary of the proposed ISAC system. To achieve the Pareto boundary, the max-min system utility function method is employed, while considering the fairness between communication users and radar targets. Subsequently, the bisection search method is employed to find a specific Pareto optimal solution by solving a series of convex feasible problems. Finally, simulation results validate that the proposed method achieves a better tradeoff between multi-user communication and multi-target sensing performance. Additionally, utilizing the tight upper bound of sensing MI as a performance metric can enhance the multi-target resolution capability and angle estimation accuracy.
△ Less
Submitted 14 May, 2024;
originally announced May 2024.
-
Fairness in Reinforcement Learning: A Survey
Authors:
Anka Reuel,
Devin Ma
Abstract:
While our understanding of fairness in machine learning has significantly progressed, our understanding of fairness in reinforcement learning (RL) remains nascent. Most of the attention has been on fairness in one-shot classification tasks; however, real-world, RL-enabled systems (e.g., autonomous vehicles) are much more complicated in that agents operate in dynamic environments over a long period…
▽ More
While our understanding of fairness in machine learning has significantly progressed, our understanding of fairness in reinforcement learning (RL) remains nascent. Most of the attention has been on fairness in one-shot classification tasks; however, real-world, RL-enabled systems (e.g., autonomous vehicles) are much more complicated in that agents operate in dynamic environments over a long period of time. To ensure the responsible development and deployment of these systems, we must better understand fairness in RL. In this paper, we survey the literature to provide the most up-to-date snapshot of the frontiers of fairness in RL. We start by reviewing where fairness considerations can arise in RL, then discuss the various definitions of fairness in RL that have been put forth thus far. We continue to highlight the methodologies researchers used to implement fairness in single- and multi-agent RL systems before showcasing the distinct application domains that fair RL has been investigated in. Finally, we critically examine gaps in the literature, such as understanding fairness in the context of RLHF, that still need to be addressed in future work to truly operationalize fair RL in real-world systems.
△ Less
Submitted 11 May, 2024;
originally announced May 2024.
-
Real-Time Pill Identification for the Visually Impaired Using Deep Learning
Authors:
Bo Dang,
Wenchao Zhao,
Yufeng Li,
Danqing Ma,
Qixuan Yu,
Elly Yijun Zhu
Abstract:
The prevalence of mobile technology offers unique opportunities for addressing healthcare challenges, especially for individuals with visual impairments. This paper explores the development and implementation of a deep learning-based mobile application designed to assist blind and visually impaired individuals in real-time pill identification. Utilizing the YOLO framework, the application aims to…
▽ More
The prevalence of mobile technology offers unique opportunities for addressing healthcare challenges, especially for individuals with visual impairments. This paper explores the development and implementation of a deep learning-based mobile application designed to assist blind and visually impaired individuals in real-time pill identification. Utilizing the YOLO framework, the application aims to accurately recognize and differentiate between various pill types through real-time image processing on mobile devices. The system incorporates Text-to- Speech (TTS) to provide immediate auditory feedback, enhancing usability and independence for visually impaired users. Our study evaluates the application's effectiveness in terms of detection accuracy and user experience, highlighting its potential to improve medication management and safety among the visually impaired community. Keywords-Deep Learning; YOLO Framework; Mobile Application; Visual Impairment; Pill Identification; Healthcare
△ Less
Submitted 7 May, 2024;
originally announced May 2024.
-
Circularly polarized light irradiated ferromagnetic MnBi$_2$Te$_4$: the long-sought ideal Weyl semimetal
Authors:
Shuai Fan,
Shengpu Huang,
Zhuo Chen,
Fangyang Zhan,
Xian-Yong Ding,
Da-Shuai Ma,
Rui Wang
Abstract:
The interaction between light and non-trivial energy band topology allows for the precise manipulation of topological quantum states, which has attracted intensive interest in condensed matter physics. In this work, using first-principles calculations, we studied the topological transition of ferromagnetic (FM) MnBi$_2$Te$_4$ upon irradiation with circularly polarized light (CPL). We revealed that…
▽ More
The interaction between light and non-trivial energy band topology allows for the precise manipulation of topological quantum states, which has attracted intensive interest in condensed matter physics. In this work, using first-principles calculations, we studied the topological transition of ferromagnetic (FM) MnBi$_2$Te$_4$ upon irradiation with circularly polarized light (CPL). We revealed that the MnBi$_2$Te$_4$ can be driven from an FM insulator to a Weyl semimetal with a minimum number of Weyl points, i.e., two Weyl points in systems without time-reversal symmetry. More importantly, in FM MnBi$_2$Te$_4$ with out-of-plane easy magnetization axis, we found that the band dispersion of the WP evolves from Type-II to Type-III and finally to Type-I when the light intensity increases. Moreover, we show that the profile of the characteristic Fermi arc of Weyl semimetal phase is sensitive to changes in light intensity, which enables efficient manipulation of the Fermi arc length of FM MnBi$_2$Te$_4$ in experiments. In addition, for FM MnBi$_2$Te$_4$ with in-plane easy magnetization axis, the system becomes a type I Weyl semimetal under CPL irradiation. With controllable band dispersion, length of Fermi arc, and minimum number of WPs, our results indicate that CPL-irradiated FM MnBi$_2$Te$_4$ is an ideal platform to study novel transport phenomena in Weyl semimetals with distinct band dispersion.
△ Less
Submitted 7 May, 2024;
originally announced May 2024.
-
3D MR Fingerprinting for Dynamic Contrast-Enhanced Imaging of Whole Mouse Brain
Authors:
Yuran Zhu,
Guanhua Wang,
Yuning Gu,
Walter Zhao,
Jiahao Lu,
Junqing Zhu,
Christina J. MacAskill,
Andrew Dupuis,
Mark A. Griswold,
Dan Ma,
Chris A. Flask,
Xin Yu
Abstract:
Quantitative MRI enables direct quantification of contrast agent concentrations in contrast-enhanced scans. However, the lengthy scan times required by conventional methods are inadequate for tracking contrast agent transport dynamically in mouse brain. We developed a 3D MR fingerprinting (MRF) method for simultaneous T1 and T2 mapping across the whole mouse brain with 4.3-min temporal resolution.…
▽ More
Quantitative MRI enables direct quantification of contrast agent concentrations in contrast-enhanced scans. However, the lengthy scan times required by conventional methods are inadequate for tracking contrast agent transport dynamically in mouse brain. We developed a 3D MR fingerprinting (MRF) method for simultaneous T1 and T2 mapping across the whole mouse brain with 4.3-min temporal resolution. We designed a 3D MRF sequence with variable acquisition segment lengths and magnetization preparations on a 9.4T preclinical MRI scanner. Model-based reconstruction approaches were employed to improve the accuracy and speed of MRF acquisition. The method's accuracy for T1 and T2 measurements was validated in vitro, while its repeatability of T1 and T2 measurements was evaluated in vivo (n=3). The utility of the 3D MRF sequence for dynamic tracking of intracisternally infused Gd-DTPA in the whole mouse brain was demonstrated (n=5). Phantom studies confirmed accurate T1 and T2 measurements by 3D MRF with an undersampling factor up to 48. Dynamic contrast-enhanced (DCE) MRF scans achieved a spatial resolution of 192 x 192 x 500 um3 and a temporal resolution of 4.3 min, allowing for the analysis and comparison of dynamic changes in concentration and transport kinetics of intracisternally infused Gd-DTPA across brain regions. The sequence also enabled highly repeatable, high-resolution T1 and T2 mapping of the whole mouse brain (192 x 192 x 250 um3) in 30 min. We present the first dynamic and multi-parametric approach for quantitatively tracking contrast agent transport in the mouse brain using 3D MRF.
△ Less
Submitted 5 August, 2024; v1 submitted 1 May, 2024;
originally announced May 2024.
-
Transformer-Based Classification Outcome Prediction for Multimodal Stroke Treatment
Authors:
Danqing Ma,
Meng Wang,
Ao Xiang,
Zongqing Qi,
Qin Yang
Abstract:
This study proposes a multi-modal fusion framework Multitrans based on the Transformer architecture and self-attention mechanism. This architecture combines the study of non-contrast computed tomography (NCCT) images and discharge diagnosis reports of patients undergoing stroke treatment, using a variety of methods based on Transformer architecture approach to predicting functional outcomes of str…
▽ More
This study proposes a multi-modal fusion framework Multitrans based on the Transformer architecture and self-attention mechanism. This architecture combines the study of non-contrast computed tomography (NCCT) images and discharge diagnosis reports of patients undergoing stroke treatment, using a variety of methods based on Transformer architecture approach to predicting functional outcomes of stroke treatment. The results show that the performance of single-modal text classification is significantly better than single-modal image classification, but the effect of multi-modal combination is better than any single modality. Although the Transformer model only performs worse on imaging data, when combined with clinical meta-diagnostic information, both can learn better complementary information and make good contributions to accurately predicting stroke treatment effects..
△ Less
Submitted 3 August, 2024; v1 submitted 19 April, 2024;
originally announced April 2024.
-
Flexible Control of Chiral Superconductivity in Optically Driven Nodal Point Superconductors with Antiferromagnetism
Authors:
Zhen Ning,
Junjie Zeng,
Da-Shuai Ma,
Dong-Hui Xu,
Rui Wang
Abstract:
Recent studies have attracted widespread attention on magnet-superconductor hybrid systems with emergent topological superconductivity. Here, we present the Floquet engineering of realistic two-dimensional topological nodal-point superconductors that are composed of antiferromagnetic monolayers in proximity to an s-wave superconductor. We show that Floquet chiral topological superconductivity aris…
▽ More
Recent studies have attracted widespread attention on magnet-superconductor hybrid systems with emergent topological superconductivity. Here, we present the Floquet engineering of realistic two-dimensional topological nodal-point superconductors that are composed of antiferromagnetic monolayers in proximity to an s-wave superconductor. We show that Floquet chiral topological superconductivity arises naturally due to light-induced breaking of the effective time-reversal symmetry. More strikingly, we find that the Floquet chiral topological superconducting phases can be flexibly controlled by irradiating elliptically polarized light, with the photon-dressed quasi-energy spectrum carrying different Chern numbers. Such optically switchable topological transition is attributed to the simultaneous creations (or annihilations) of valley pairs. Our findings provide a feasible approach for achieving the Floquet chiral topological superconductivity with flexible tunability, which would draw extensive attention in experiments.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Cramer-Rao Bounds for Near-Field Sensing: A Generic Modular Architecture
Authors:
Chunwei Meng,
Dingyou Ma,
Xu Chen,
Zhiyong Feng,
Yuanwei Liu
Abstract:
A generic modular array architecture is proposed, featuring uniform/non-uniform subarray layouts that allows for flexible deployment. The bistatic near-field sensing system is considered, where the target is located in the near-field of the whole modular array and the far-field of each subarray. Then, the closed-form expressions of Cramer-Rao bounds (CRBs) for range and angle estimations are deriv…
▽ More
A generic modular array architecture is proposed, featuring uniform/non-uniform subarray layouts that allows for flexible deployment. The bistatic near-field sensing system is considered, where the target is located in the near-field of the whole modular array and the far-field of each subarray. Then, the closed-form expressions of Cramer-Rao bounds (CRBs) for range and angle estimations are derived based on the hybrid spherical and planar wave model (HSPM). Simulation results validate the accuracy of the derived closed-form CRBs and demonstrate that: i) The HSPM with varying angles of arrival (AoAs) between subarrays can reduce the CRB for range estimation compared to the traditional HSPM with shared AoA; and ii) The proposed generic modular architecture with subarrays positioned closer to the edges can significantly reduce the CRBs compared to the traditional modular architecture with uniform subarray layout, when the array aperture is fixed.
△ Less
Submitted 11 April, 2024;
originally announced April 2024.
-
Surface variation analysis of freeform optical systems over surface frequency bands for prescribed wavefront errors
Authors:
Rundong Fan,
Shili Wei,
Huiru JI,
Zhuang Qian,
Hao Tan,
Yan Mo,
Donglin MA
Abstract:
The surface errors of freeform surfaces reflect the manufacturing complexities and significantly impact the feasibility of processing designed optical systems. With multiple degrees of freedom, freeform surfaces pose challenges in surface tolerance analysis in the field. Nevertheless, current research has neglected the influence of surface slopes on the directions of ray propagation. A sudden alte…
▽ More
The surface errors of freeform surfaces reflect the manufacturing complexities and significantly impact the feasibility of processing designed optical systems. With multiple degrees of freedom, freeform surfaces pose challenges in surface tolerance analysis in the field. Nevertheless, current research has neglected the influence of surface slopes on the directions of ray propagation. A sudden alteration in the surface slope will lead to a corresponding abrupt shift in the wavefront, even when the change in surface sag is minimal. Moreover, within the realm of freeform surface manufacturing, variation in surface slope across different frequency bands may give rise to unique surface variation. Within the context of this study, we propose a tolerance analysis method to analyze surface variation in freeform surfaces considering surface frequency band slopes based on real ray data. This approach utilizes real ray data to rapidly evaluate surface variation within a specified frequency band of surface slopes. Crucially, our proposed method yields the capability to obtain system surface variation with significant wavefront aberration, in contrast to previous methodologies. The feasibility and advantages of this framework are assessed by analyzing a single-mirror system with a single field and an off-axis two-mirror system. We expect to integrate the proposed methodology with freeform surface design and manufacturing, thereby expanding the scope of freeform optics.
△ Less
Submitted 27 March, 2024;
originally announced March 2024.
-
Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback
Authors:
Hongshen Xu,
Zichen Zhu,
Situo Zhang,
Da Ma,
Shuai Fan,
Lu Chen,
Kai Yu
Abstract:
Large Language Models (LLMs) often generate erroneous outputs, known as hallucinations, due to their limitations in discerning questions beyond their knowledge scope. While addressing hallucination has been a focal point in research, previous efforts primarily concentrate on enhancing correctness without giving due consideration to the significance of rejection mechanisms. In this paper, we conduc…
▽ More
Large Language Models (LLMs) often generate erroneous outputs, known as hallucinations, due to their limitations in discerning questions beyond their knowledge scope. While addressing hallucination has been a focal point in research, previous efforts primarily concentrate on enhancing correctness without giving due consideration to the significance of rejection mechanisms. In this paper, we conduct a comprehensive examination of the role of rejection, introducing the notion of model reliability along with corresponding metrics. These metrics measure the model's ability to provide accurate responses while adeptly rejecting questions exceeding its knowledge boundaries, thereby minimizing hallucinations. To improve the inherent reliability of LLMs, we present a novel alignment framework called Reinforcement Learning from Knowledge Feedback (RLKF). RLKF leverages knowledge feedback to dynamically determine the model's knowledge boundary and trains a reliable reward model to encourage the refusal of out-of-knowledge questions. Experimental results on mathematical questions affirm the substantial efficacy of RLKF in significantly enhancing LLM reliability.
△ Less
Submitted 8 August, 2024; v1 submitted 27 March, 2024;
originally announced March 2024.
-
MA4DIV: Multi-Agent Reinforcement Learning for Search Result Diversification
Authors:
Yiqun Chen,
Jiaxin Mao,
Yi Zhang,
Dehong Ma,
Long Xia,
Jun Fan,
Daiting Shi,
Zhicong Cheng,
Simiu Gu,
Dawei Yin
Abstract:
The objective of search result diversification (SRD) is to ensure that selected documents cover as many different subtopics as possible. Existing methods primarily utilize a paradigm of "greedy selection", i.e., selecting one document with the highest diversity score at a time. These approaches tend to be inefficient and are easily trapped in a suboptimal state. In addition, some other methods aim…
▽ More
The objective of search result diversification (SRD) is to ensure that selected documents cover as many different subtopics as possible. Existing methods primarily utilize a paradigm of "greedy selection", i.e., selecting one document with the highest diversity score at a time. These approaches tend to be inefficient and are easily trapped in a suboptimal state. In addition, some other methods aim to approximately optimize the diversity metric, such as $α$-NDCG, but the results still remain suboptimal. To address these challenges, we introduce Multi-Agent reinforcement learning (MARL) for search result DIVersity, which called MA4DIV. In this approach, each document is an agent and the search result diversification is modeled as a cooperative task among multiple agents. This approach allows for directly optimizing the diversity metrics, such as $α$-NDCG, while achieving high training efficiency. We conducted preliminary experiments on public TREC datasets to demonstrate the effectiveness and potential of MA4DIV. Considering the limited number of queries in public TREC datasets, we construct a large-scale dataset from industry sources and show that MA4DIV achieves substantial improvements in both effectiveness and efficiency than existing baselines on a industrial scale dataset.
△ Less
Submitted 27 March, 2024; v1 submitted 26 March, 2024;
originally announced March 2024.
-
Utilizing the LightGBM Algorithm for Operator User Credit Assessment Research
Authors:
Shaojie Li,
Xinqi Dong,
Danqing Ma,
Bo Dang,
Hengyi Zang,
Yulu Gong
Abstract:
Mobile Internet user credit assessment is an important way for communication operators to establish decisions and formulate measures, and it is also a guarantee for operators to obtain expected benefits. However, credit evaluation methods have long been monopolized by financial industries such as banks and credit. As supporters and providers of platform network technology and network resources, co…
▽ More
Mobile Internet user credit assessment is an important way for communication operators to establish decisions and formulate measures, and it is also a guarantee for operators to obtain expected benefits. However, credit evaluation methods have long been monopolized by financial industries such as banks and credit. As supporters and providers of platform network technology and network resources, communication operators are also builders and maintainers of communication networks. Internet data improves the user's credit evaluation strategy. This paper uses the massive data provided by communication operators to carry out research on the operator's user credit evaluation model based on the fusion LightGBM algorithm. First, for the massive data related to user evaluation provided by operators, key features are extracted by data preprocessing and feature engineering methods, and a multi-dimensional feature set with statistical significance is constructed; then, linear regression, decision tree, LightGBM, and other machine learning algorithms build multiple basic models to find the best basic model; finally, integrates Averaging, Voting, Blending, Stacking and other integrated algorithms to refine multiple fusion models, and finally establish the most suitable fusion model for operator user evaluation.
△ Less
Submitted 21 March, 2024;
originally announced March 2024.
-
Fostc3net:A Lightweight YOLOv5 Based On the Network Structure Optimization
Authors:
Danqing Ma,
Shaojie Li,
Bo Dang,
Hengyi Zang,
Xinqi Dong
Abstract:
Transmission line detection technology is crucial for automatic monitoring and ensuring the safety of electrical facilities. The YOLOv5 series is currently one of the most advanced and widely used methods for object detection. However, it faces inherent challenges, such as high computational load on devices and insufficient detection accuracy. To address these concerns, this paper presents an enha…
▽ More
Transmission line detection technology is crucial for automatic monitoring and ensuring the safety of electrical facilities. The YOLOv5 series is currently one of the most advanced and widely used methods for object detection. However, it faces inherent challenges, such as high computational load on devices and insufficient detection accuracy. To address these concerns, this paper presents an enhanced lightweight YOLOv5 technique customized for mobile devices, specifically intended for identifying objects associated with transmission lines. The C3Ghost module is integrated into the convolutional network of YOLOv5 to reduce floating point operations per second (FLOPs) in the feature channel fusion process and improve feature expression performance. In addition, a FasterNet module is introduced to replace the c3 module in the YOLOv5 Backbone. The FasterNet module uses Partial Convolutions to process only a portion of the input channels, improving feature extraction efficiency and reducing computational overhead. To address the imbalance between simple and challenging samples in the dataset and the diversity of aspect ratios of bounding boxes, the wIoU v3 LOSS is adopted as the loss function. To validate the performance of the proposed approach, Experiments are conducted on a custom dataset of transmission line poles. The results show that the proposed model achieves a 1% increase in detection accuracy, a 13% reduction in FLOPs, and a 26% decrease in model parameters compared to the existing YOLOv5.In the ablation experiment, it was also discovered that while the Fastnet module and the CSghost module improved the precision of the original YOLOv5 baseline model, they caused a decrease in the [email protected] metric. However, the improvement of the wIoUv3 loss function significantly mitigated the decline of the [email protected] metric.
△ Less
Submitted 20 March, 2024;
originally announced March 2024.
-
EAS-SNN: End-to-End Adaptive Sampling and Representation for Event-based Detection with Recurrent Spiking Neural Networks
Authors:
Ziming Wang,
Ziling Wang,
Huaning Li,
Lang Qin,
Runhao Jiang,
De Ma,
Huajin Tang
Abstract:
Event cameras, with their high dynamic range and temporal resolution, are ideally suited for object detection, especially under scenarios with motion blur and challenging lighting conditions. However, while most existing approaches prioritize optimizing spatiotemporal representations with advanced detection backbones and early aggregation functions, the crucial issue of adaptive event sampling rem…
▽ More
Event cameras, with their high dynamic range and temporal resolution, are ideally suited for object detection, especially under scenarios with motion blur and challenging lighting conditions. However, while most existing approaches prioritize optimizing spatiotemporal representations with advanced detection backbones and early aggregation functions, the crucial issue of adaptive event sampling remains largely unaddressed. Spiking Neural Networks (SNNs), which operate on an event-driven paradigm through sparse spike communication, emerge as a natural fit for addressing this challenge. In this study, we discover that the neural dynamics of spiking neurons align closely with the behavior of an ideal temporal event sampler. Motivated by this insight, we propose a novel adaptive sampling module that leverages recurrent convolutional SNNs enhanced with temporal memory, facilitating a fully end-to-end learnable framework for event-based detection. Additionally, we introduce Residual Potential Dropout (RPD) and Spike-Aware Training (SAT) to regulate potential distribution and address performance degradation encountered in spike-based sampling modules. Empirical evaluation on neuromorphic detection datasets demonstrates that our approach outperforms existing state-of-the-art spike-based methods with significantly fewer parameters and time steps. For instance, our method yields a 4.4\% mAP improvement on the Gen1 dataset, while requiring 38\% fewer parameters and only three time steps. Moreover, the applicability and effectiveness of our adaptive sampling methodology extend beyond SNNs, as demonstrated through further validation on conventional non-spiking models. Code is available at https://github.com/Windere/EAS-SNN.
△ Less
Submitted 24 August, 2024; v1 submitted 19 March, 2024;
originally announced March 2024.
-
A maximum penalised likelihood approach for semiparametric accelerated failure time models with time-varying covariates and partly interval censoring
Authors:
Aishwarya Bhaskaran,
Ding Ma,
Benoit Liquet,
Angela Hong,
Serigne N Lo,
Stephane Heritier,
Jun Ma
Abstract:
Accelerated failure time (AFT) models are frequently used for modelling survival data. This approach is attractive as it quantifies the direct relationship between the time until an event occurs and various covariates. It asserts that the failure times experience either acceleration or deceleration through a multiplicative factor when these covariates are present. While existing literature provide…
▽ More
Accelerated failure time (AFT) models are frequently used for modelling survival data. This approach is attractive as it quantifies the direct relationship between the time until an event occurs and various covariates. It asserts that the failure times experience either acceleration or deceleration through a multiplicative factor when these covariates are present. While existing literature provides numerous methods for fitting AFT models with time-fixed covariates, adapting these approaches to scenarios involving both time-varying covariates and partly interval-censored data remains challenging. In this paper, we introduce a maximum penalised likelihood approach to fit a semiparametric AFT model. This method, designed for survival data with partly interval-censored failure times, accommodates both time-fixed and time-varying covariates. We utilise Gaussian basis functions to construct a smooth approximation of the nonparametric baseline hazard and fit the model via a constrained optimisation approach. To illustrate the effectiveness of our proposed method, we conduct a comprehensive simulation study. We also present an implementation of our approach on a randomised clinical trial dataset on advanced melanoma patients.
△ Less
Submitted 18 March, 2024;
originally announced March 2024.
-
DiTMoS: Delving into Diverse Tiny-Model Selection on Microcontrollers
Authors:
Xiao Ma,
Shengfeng He,
Hezhe Qiao,
Dong Ma
Abstract:
Enabling efficient and accurate deep neural network (DNN) inference on microcontrollers is non-trivial due to the constrained on-chip resources. Current methodologies primarily focus on compressing larger models yet at the expense of model accuracy. In this paper, we rethink the problem from the inverse perspective by constructing small/weak models directly and improving their accuracy. Thus, we i…
▽ More
Enabling efficient and accurate deep neural network (DNN) inference on microcontrollers is non-trivial due to the constrained on-chip resources. Current methodologies primarily focus on compressing larger models yet at the expense of model accuracy. In this paper, we rethink the problem from the inverse perspective by constructing small/weak models directly and improving their accuracy. Thus, we introduce DiTMoS, a novel DNN training and inference framework with a selector-classifiers architecture, where the selector routes each input sample to the appropriate classifier for classification. DiTMoS is grounded on a key insight: a composition of weak models can exhibit high diversity and the union of them can significantly boost the accuracy upper bound. To approach the upper bound, DiTMoS introduces three strategies including diverse training data splitting to increase the classifiers' diversity, adversarial selector-classifiers training to ensure synergistic interactions thereby maximizing their complementarity, and heterogeneous feature aggregation to improve the capacity of classifiers. We further propose a network slicing technique to alleviate the extra memory overhead incurred by feature aggregation. We deploy DiTMoS on the Neucleo STM32F767ZI board and evaluate it based on three time-series datasets for human activity recognition, keywords spotting, and emotion recognition, respectively. The experiment results manifest that: (a) DiTMoS achieves up to 13.4% accuracy improvement compared to the best baseline; (b) network slicing almost completely eliminates the memory overhead incurred by feature aggregation with a marginal increase of latency.
△ Less
Submitted 13 March, 2024;
originally announced March 2024.
-
A Multimodal Fusion Network For Student Emotion Recognition Based on Transformer and Tensor Product
Authors:
Ao Xiang,
Zongqing Qi,
Han Wang,
Qin Yang,
Danqing Ma
Abstract:
This paper introduces a new multi-modal model based on the Transformer architecture and tensor product fusion strategy, combining BERT's text vectors and ViT's image vectors to classify students' psychological conditions, with an accuracy of 93.65%. The purpose of the study is to accurately analyze the mental health status of students from various data sources. This paper discusses modal fusion me…
▽ More
This paper introduces a new multi-modal model based on the Transformer architecture and tensor product fusion strategy, combining BERT's text vectors and ViT's image vectors to classify students' psychological conditions, with an accuracy of 93.65%. The purpose of the study is to accurately analyze the mental health status of students from various data sources. This paper discusses modal fusion methods, including early, late and intermediate fusion, to overcome the challenges of integrating multi-modal information. Ablation studies compare the performance of different models and fusion techniques, showing that the proposed model outperforms existing methods such as CLIP and ViLBERT in terms of accuracy and inference speed. Conclusions indicate that while this model has significant advantages in emotion recognition, its potential to incorporate other data modalities provides areas for future research.
△ Less
Submitted 19 April, 2024; v1 submitted 13 March, 2024;
originally announced March 2024.
-
Improved YOLOv5 Based on Attention Mechanism and FasterNet for Foreign Object Detection on Railway and Airway tracks
Authors:
Zongqing Qi,
Danqing Ma,
Jingyu Xu,
Ao Xiang,
Hedi Qu
Abstract:
In recent years, there have been frequent incidents of foreign objects intruding into railway and Airport runways. These objects can include pedestrians, vehicles, animals, and debris. This paper introduces an improved YOLOv5 architecture incorporating FasterNet and attention mechanisms to enhance the detection of foreign objects on railways and Airport runways. This study proposes a new dataset,…
▽ More
In recent years, there have been frequent incidents of foreign objects intruding into railway and Airport runways. These objects can include pedestrians, vehicles, animals, and debris. This paper introduces an improved YOLOv5 architecture incorporating FasterNet and attention mechanisms to enhance the detection of foreign objects on railways and Airport runways. This study proposes a new dataset, AARFOD (Aero and Rail Foreign Object Detection), which combines two public datasets for detecting foreign objects in aviation and railway systems.The dataset aims to improve the recognition capabilities of foreign object targets. Experimental results on this large dataset have demonstrated significant performance improvements of the proposed model over the baseline YOLOv5 model, reducing computational requirements.Improved YOLO model shows a significant improvement in precision by 1.2%, recall rate by 1.0%, and [email protected] by 0.6%, while [email protected] remained unchanged. The parameters were reduced by approximately 25.12%, and GFLOPs were reduced by about 10.63%. In the ablation experiment, it is found that the FasterNet module can significantly reduce the number of parameters of the model, and the reference of the attention mechanism can slow down the performance loss caused by lightweight.
△ Less
Submitted 13 March, 2024;
originally announced March 2024.
-
Improving Event Definition Following For Zero-Shot Event Detection
Authors:
Zefan Cai,
Po-Nien Kung,
Ashima Suvarna,
Mingyu Derek Ma,
Hritik Bansal,
Baobao Chang,
P. Jeffrey Brantingham,
Wei Wang,
Nanyun Peng
Abstract:
Existing approaches on zero-shot event detection usually train models on datasets annotated with known event types, and prompt them with unseen event definitions. These approaches yield sporadic successes, yet generally fall short of expectations. In this work, we aim to improve zero-shot event detection by training models to better follow event definitions. We hypothesize that a diverse set of ev…
▽ More
Existing approaches on zero-shot event detection usually train models on datasets annotated with known event types, and prompt them with unseen event definitions. These approaches yield sporadic successes, yet generally fall short of expectations. In this work, we aim to improve zero-shot event detection by training models to better follow event definitions. We hypothesize that a diverse set of event types and definitions are the key for models to learn to follow event definitions while existing event extraction datasets focus on annotating many high-quality examples for a few event types. To verify our hypothesis, we construct an automatically generated Diverse Event Definition (DivED) dataset and conduct comparative studies. Our experiments reveal that a large number of event types (200) and diverse event definitions can significantly boost event extraction performance; on the other hand, the performance does not scale with over ten examples per event type. Beyond scaling, we incorporate event ontology information and hard-negative samples during training, further boosting the performance. Based on these findings, we fine-tuned a LLaMA-2-7B model on our DivED dataset, yielding performance that surpasses SOTA large language models like GPT-3.5 across three open benchmarks on zero-shot event detection.
△ Less
Submitted 4 March, 2024;
originally announced March 2024.
-
Hierarchical Multimodal Pre-training for Visually Rich Webpage Understanding
Authors:
Hongshen Xu,
Lu Chen,
Zihan Zhao,
Da Ma,
Ruisheng Cao,
Zichen Zhu,
Kai Yu
Abstract:
The growing prevalence of visually rich documents, such as webpages and scanned/digital-born documents (images, PDFs, etc.), has led to increased interest in automatic document understanding and information extraction across academia and industry. Although various document modalities, including image, text, layout, and structure, facilitate human information retrieval, the interconnected nature of…
▽ More
The growing prevalence of visually rich documents, such as webpages and scanned/digital-born documents (images, PDFs, etc.), has led to increased interest in automatic document understanding and information extraction across academia and industry. Although various document modalities, including image, text, layout, and structure, facilitate human information retrieval, the interconnected nature of these modalities presents challenges for neural networks. In this paper, we introduce WebLM, a multimodal pre-training network designed to address the limitations of solely modeling text and structure modalities of HTML in webpages. Instead of processing document images as unified natural images, WebLM integrates the hierarchical structure of document images to enhance the understanding of markup-language-based documents. Additionally, we propose several pre-training tasks to model the interaction among text, structure, and image modalities effectively. Empirical results demonstrate that the pre-trained WebLM significantly surpasses previous state-of-the-art pre-trained models across several webpage understanding tasks. The pre-trained models and code are available at https://github.com/X-LANCE/weblm.
△ Less
Submitted 28 February, 2024;
originally announced February 2024.