Showing 1–50 of 387 results for author: Zeng, L

Excellent and CO$_2$$_{0.85}$Nd$_{0.1}$Cu$_{0.05}$O$_{2-δ}$-Nd$_x$Sr$_{1-x}$Fe$_{1-y}$Cu$_y$O$_{3-δ}$ dual-phase oxygen transport membranes

Authors: Chao Zhang, Yue Zhu, Xiaopeng Wang, Yanhao Huang, Lingyong Zeng, Kuan Li, Peifeng Yu, Kangwang Wang, Longfu Li, Zaichen Xiang, Rui Chen, Xuefeng Zhu, Huixia Luo

Abstract: Oxygen transport membranes(OTMs)have provided great opportunities in the last decades but are suffering from the trade-off effect between stability and oxygen permeability. Here, we report a group of new planar dual-phase mixed ionic-electronic conducting (MIEC) OTMs consisting of CO$_2$$_{0.85}$Nd$_{0.1}$Cu$_{0.05}$O$_2$ (CNCO) and Nd$_x$Sr$_{1-x}$Fe$_{1-y}$Cu$_y$O$_3$(NSFCO; $x = 0.4, 0.6$;… ▽ More Oxygen transport membranes(OTMs)have provided great opportunities in the last decades but are suffering from the trade-off effect between stability and oxygen permeability. Here, we report a group of new planar dual-phase mixed ionic-electronic conducting (MIEC) OTMs consisting of CO$_2$$_{0.85}$Nd$_{0.1}$Cu$_{0.05}$O$_2$ (CNCO) and Nd$_x$Sr$_{1-x}$Fe$_{1-y}$Cu$_y$O$_3$(NSFCO; $x = 0.4, 0.6$; $y = 0.05, 0.1$) phases, showing excellent oxygen permeability while comparable CO$_2$-resistant stability. The substitution of Cu as a bifunctional additive decreases the sintering temperature and enhances bulk diffusion and oxygen permeability with the co-doping of Nd.The oxygen permeation fluxes reached 2.62 and 1.52 mL min$^{-1}$ cm$^{-2}$ at 1000$^\circ$C through the optimal 60wt%Ce0.85Nd0.1Cu0.05O2-40wt%Nd0.4Sr0.6Fe0.9Cu0.1O3 (CNCO-NSFCO41) composition with He and CO$_2$ sweeping, respectively, higher than all reported dense dual-phase OTMs. Such excellent CO$_2$-tolerant permeability meets the needs of potential industrial applications. Analysis with Zhu's oxygen permeation model shows lower bulk diffusion resistance of CNCO-NSFCO41 than that of reported 60wt%Ce0.85Pr0.1Cu0.05O2-40wt%Pr0.4Sr0.6Fe0.9Cu0.1O3(CPCO-PSFCO41)and more limitation by the interfacial exchange at high temperature. All the prepared OTMs also show good long-term stability over 100 hours in both atmospheres. Our results confirm the excellent oxygen permeability and stability under a high-concentration CO2 atmosphere, providing a material candidate for CO2 capture in oxyfuel combustion. △ Less

Submitted 22 August, 2024; originally announced August 2024.

Comments: 36 pages, 6 figures

Journal ref: Journal of Membrane Science, 2024,696,122485

Mapping Hydrogen Evolution Activity Trends of V-based A15 Superconducting Alloys

Authors: Peifeng Yu, Jie Zhan, Xiaobing Zhang, Kangwang Wang, Lingyong Zeng, Kuan Li, Chao Zhang, Longfu Li, Ying Liang, Kai Yan, Yan Sun, Huixia Luo

Abstract: Exploring high-efficiency and low-cost electrocatalysts is valuable for water-splitting technologies. Recently, Si-group compounds have attracted increasing attention in electrocatalysis, considering the abundant Si-group elements on Earth. However, Si-group compounds for HER electrocatalysis have not been systematically studied. In this study, we unveil the activity trends of non-noble metal cata… ▽ More Exploring high-efficiency and low-cost electrocatalysts is valuable for water-splitting technologies. Recently, Si-group compounds have attracted increasing attention in electrocatalysis, considering the abundant Si-group elements on Earth. However, Si-group compounds for HER electrocatalysis have not been systematically studied. In this study, we unveil the activity trends of non-noble metal catalyst A15-type V3M (i.e., V3Si, V3Ge, and V3Sn) superconductors and show that V3Si is the most efficient HER catalyst because of the high electronic conductivity and suitable d-band center. Among them, the V3Si only requires 33.4 mV to reach 10 mA cm-2, and only 57.6 mV and 114.6 mV are required to attain a high current density of 100 mA cm-2 and 500 mA cm-2, respectively. These low overpotentials are close to the 34.3 mV at 10 mA cm-2 of state-of-art Pt/C (20 %) but superior to 168.5 mV of Pt/C (20 %) at 100 mA cm-2. Furthermore, the V3Si illustrates exceptional durability with no obvious decay in the 120 h at the different current densities (i.e., 10 - 250 mA cm-2). The excellent HER activity of V3Si alloy can be ascribed to the synergies of superior electronic conductivity and suitable d-band center. Moreover, DFT calculations reveal that the absolute hydrogen adsorption Gibbs free energy is decreased after introducing the V to Si. Beyond offering a stable and high-performance electrocatalyst in an acidic medium, this work inspires the rational design of desirable silicide electrocatalysts. △ Less

Submitted 22 August, 2024; originally announced August 2024.

Comments: 25 pages, 5 figures

Journal ref: Chemical Engineering Journal,2024, 488, 150961

Structural and Superconducting Properties in the Te-doped Spinel CuRh2Se4

Authors: Kuan Li, Lingyong Zeng, Longfu Li, Rui Chen, Peifeng Yu, Kangwang Wang, Chao Zhang, Zaichen Xiang, Huixia Luo

Abstract: In this paper, we discuss the impact of tellurium (Te) doping on the spinel superconductor CuRh2Se4. We conducted a comprehensive evaluation of the structural and superconducting properties of the system using various techniques, including X-ray diffraction (XRD), resistivity, magnetization, and specific heat measurements. Based on our XRD analysis, we found that the spinel superconductor CuRh2Se4… ▽ More In this paper, we discuss the impact of tellurium (Te) doping on the spinel superconductor CuRh2Se4. We conducted a comprehensive evaluation of the structural and superconducting properties of the system using various techniques, including X-ray diffraction (XRD), resistivity, magnetization, and specific heat measurements. Based on our XRD analysis, we found that the spinel superconductor CuRh2Se4-xTex crystallizes in the space group Fd3m(227) with x in the region of 0 to 0.28, while the layered compound CuRh2Se4-xTex crystallizes in the space group P3m1 (164) with x in the region of 2.8 to 4.0. The upper critical magnetic field can be increased from 0.95(2) T for CuRh2Se4 to 3.44(1) T for CuRh2Se3.72Te0.28 by doping with elemental Te. However, the layered compound CuRh2Se4-xTex did not exhibit superconducting properties. Besides, the specific heat measurements of CuRh2Se4-xTex (x = 0, 0.1, 0.28) indicate that the Te element doping affects the electronic structure and interactions of the material and breaks the stability of the superconducting pairing, which leads to a decrease in the Tc. Finally, we show the electronic phase diagram of Tc with Te doping to summarise our findings. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Comments: 25 pages, 6 figures, 1 table

Journal ref: Journal of Alloys and Compounds, 2024, 995, 174756

Revealing the nontrivial topological surface states of catalysts for effective photochemical carbon dioxide conversion

Authors: Kangwang Wang, Longfu Li, Peifeng Yu, Nannan Tang, Lingyong Zeng, Kuan Li, Chao Zhang, Rui Chen, Zaichen Xiang, Huichao Wang, Yongqing Cai, Kai Yan, Huixia Luo

Abstract: Topological semimetals with protected surface states mark a new paradigm of research beyond the early landmarks of band-structure engineering, allowing fabrication of efficient catalyst to harness the rich metallic surface states to activate specific chemical processes. Herein, we demonstrate a facile solid-phase method for in-situ doping of Ir at the Os sites in the Os3Sn7, an alloy with topologi… ▽ More Topological semimetals with protected surface states mark a new paradigm of research beyond the early landmarks of band-structure engineering, allowing fabrication of efficient catalyst to harness the rich metallic surface states to activate specific chemical processes. Herein, we demonstrate a facile solid-phase method for in-situ doping of Ir at the Os sites in the Os3Sn7, an alloy with topological states, which significantly improves the photocatalytic performance for the reduction of CO2 to CO and CH4. Experimental evidence combined with theoretical calculations reveal that the nontrivial topological surface states greatly accelerate charge-separation/electron-enrichment and adsorption/activation of CO2 molecules, rendering highly efficient reaction channels to stimulate the formation of *COOH and *CO, as well CHO*. This work shows the promise of achieving high photocatalytic performances with synthesizing topological catalysts and provides hints on the design of novel topological catalysts with superior photoactivity towards the CO2 reduction reaction. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Comments: 33 Pages, 6 Figures, 1 Table

Journal ref: Applied Catalysis B: Environment and Energy,2024,358,124428

Non-trivial Topological Surface States Regulation of 1T-OsCoTe$_2$ Enables Selective C-C Coupling for Highly Efficient Photochemical CO$_2$ Reduction Toward C$_{2+}$ hydrocarbons

Authors: Kangwang Wang, Mingjie Wu, Peifeng Yu, Hector F. Garces, Ying Liang, Longfu Li, Lingyong Zeng, Kuan Li, Chao Zhang, Kai Yan, Huixia Luo

Abstract: Despite ongoing research, the rational design of nontrivial topological semimetal surface states for the selective photocatalytic CO$_2$ conversion into valuable products remains full of challenges. Herein, we present the synthesis of 1T-OsCoTe$_2$ for the photoreduction upgrading of CO$_2$ to tricarbon alkane C$_3$H$_8$,by the integration of experimental work and theory calculation. Experimental… ▽ More Despite ongoing research, the rational design of nontrivial topological semimetal surface states for the selective photocatalytic CO$_2$ conversion into valuable products remains full of challenges. Herein, we present the synthesis of 1T-OsCoTe$_2$ for the photoreduction upgrading of CO$_2$ to tricarbon alkane C$_3$H$_8$,by the integration of experimental work and theory calculation. Experimental studies suggested a high electron based selectivity of 71.2% for C$_3$H$_8$ and an internal quantum efficiency of 54.6% at 380 nm. In-situ X-ray photoelectron spectroscopy and X-ray absorption fine structure spectroscopy demonstrated that Co and Os atoms coordinated with Te atoms enable an efficient Os-Te-Co electron transfer to activate the generation of *CH$_3$,*CHOCO and *CH$_2$OCOCO. Density functional theory calculations further confirmed Os-Te-Co electron bridging on the improved CO$_2$ conversion kinetics. To our knowledge, this is the first report suggesting the role of Os atoms in accelerating the photocatalytic CO$_2$ conversion activity of the topological semimetal 1T-OsCoTe$_2$. △ Less

Submitted 21 August, 2024; originally announced August 2024.

Comments: 31 pages, 6 Figures

Journal ref: Applied Catalysis B: Environment and Energy,2024,352,124058

Pluto and Charon: A Time and Memory Efficient Collaborative Edge AI Framework for Personal LLMs Fine-Tuning

Authors: Bei Ouyang, Shengyuan Ye, Liekang Zeng, Tianyi Qian, Jingyi Li, Xu Chen

Abstract: Large language models (LLMs) have unlocked a plethora of powerful applications at the network edge, such as intelligent personal assistants. Data privacy and security concerns have prompted a shift towards edge-based fine-tuning of personal LLMs, away from cloud reliance. However, this raises issues of computational intensity and resource scarcity, hindering training efficiency and feasibility. Wh… ▽ More Large language models (LLMs) have unlocked a plethora of powerful applications at the network edge, such as intelligent personal assistants. Data privacy and security concerns have prompted a shift towards edge-based fine-tuning of personal LLMs, away from cloud reliance. However, this raises issues of computational intensity and resource scarcity, hindering training efficiency and feasibility. While current studies investigate parameter-efficient fine-tuning (PEFT) techniques to mitigate resource constraints, our analysis indicates that these techniques are not sufficiently resource-efficient for edge devices. To tackle these challenges, we propose Pluto and Charon (PAC), a time and memory efficient collaborative edge AI framework for personal LLMs fine-tuning. PAC breaks the resource wall of personal LLMs fine-tuning with a sophisticated algorithm-system co-design. (1) Algorithmically, PAC implements a personal LLMs fine-tuning technique that is efficient in terms of parameters, time, and memory. It utilizes Parallel Adapters to circumvent the need for a full backward pass through the LLM backbone. Additionally, an activation cache mechanism further streamlining the process by negating the necessity for repeated forward passes across multiple epochs. (2) Systematically, PAC leverages edge devices in close proximity, pooling them as a collective resource for in-situ personal LLMs fine-tuning, utilizing a hybrid data and pipeline parallelism to orchestrate distributed training. The use of the activation cache eliminates the need for forward pass through the LLM backbone,enabling exclusive fine-tuning of the Parallel Adapters using data parallelism. Extensive evaluation based on prototype implementation demonstrates that PAC remarkably outperforms state-of-the-art approaches, achieving up to 8.64x end-to-end speedup and up to 88.16% reduction in memory footprint. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Comments: Accepted by The 53rd International Conference on Parallel Processing (ICPP'24)

Asteroid: Resource-Efficient Hybrid Pipeline Parallelism for Collaborative DNN Training on Heterogeneous Edge Devices

Authors: Shengyuan Ye, Liekang Zeng, Xiaowen Chu, Guoliang Xing, Xu Chen

Abstract: On-device Deep Neural Network (DNN) training has been recognized as crucial for privacy-preserving machine learning at the edge. However, the intensive training workload and limited onboard computing resources pose significant challenges to the availability and efficiency of model training. While existing works address these challenges through native resource management optimization, we instead le… ▽ More On-device Deep Neural Network (DNN) training has been recognized as crucial for privacy-preserving machine learning at the edge. However, the intensive training workload and limited onboard computing resources pose significant challenges to the availability and efficiency of model training. While existing works address these challenges through native resource management optimization, we instead leverage our observation that edge environments usually comprise a rich set of accompanying trusted edge devices with idle resources beyond a single terminal. We propose Asteroid, a distributed edge training system that breaks the resource walls across heterogeneous edge devices for efficient model training acceleration. Asteroid adopts a hybrid pipeline parallelism to orchestrate distributed training, along with a judicious parallelism planning for maximizing throughput under certain resource constraints. Furthermore, a fault-tolerant yet lightweight pipeline replay mechanism is developed to tame the device-level dynamics for training robustness and performance stability. We implement Asteroid on heterogeneous edge devices with both vision and language models, demonstrating up to 12.2x faster training than conventional parallelism methods and 2.1x faster than state-of-the-art hybrid parallelism methods through evaluations. Furthermore, Asteroid can recover training pipeline 14x faster than baseline methods while preserving comparable throughput despite unexpected device exiting and failure. △ Less

Submitted 15 August, 2024; originally announced August 2024.

Comments: Accepted by The 30th Annual International Conference on Mobile Computing and Networking (MobiCom'24)

Freehand Sketch Generation from Mechanical Components

Authors: Zhichao Liao, Di Huang, Heming Fang, Yue Ma, Fengyuan Piao, Xinghui Li, Long Zeng, Pingfa Feng

Abstract: Drawing freehand sketches of mechanical components on multimedia devices for AI-based engineering modeling has become a new trend. However, its development is being impeded because existing works cannot produce suitable sketches for data-driven research. These works either generate sketches lacking a freehand style or utilize generative models not originally designed for this task resulting in poo… ▽ More Drawing freehand sketches of mechanical components on multimedia devices for AI-based engineering modeling has become a new trend. However, its development is being impeded because existing works cannot produce suitable sketches for data-driven research. These works either generate sketches lacking a freehand style or utilize generative models not originally designed for this task resulting in poor effectiveness. To address this issue, we design a two-stage generative framework mimicking the human sketching behavior pattern, called MSFormer, which is the first time to produce humanoid freehand sketches tailored for mechanical components. The first stage employs Open CASCADE technology to obtain multi-view contour sketches from mechanical components, filtering perturbing signals for the ensuing generation process. Meanwhile, we design a view selector to simulate viewpoint selection tasks during human sketching for picking out information-rich sketches. The second stage translates contour sketches into freehand sketches by a transformer-based generator. To retain essential modeling features as much as possible and rationalize stroke distribution, we introduce a novel edge-constraint stroke initialization. Furthermore, we utilize a CLIP vision encoder and a new loss function incorporating the Hausdorff distance to enhance the generalizability and robustness of the model. Extensive experiments demonstrate that our approach achieves state-of-the-art performance for generating freehand sketches in the mechanical domain. Project page: https://mcfreeskegen.github.io . △ Less

Submitted 21 August, 2024; v1 submitted 12 August, 2024; originally announced August 2024.

Comments: Published at ACM Multimedia (ACM MM) 2024

Suppression of Edge Localized Modes in ITER Baseline Scenario in EAST using Edge Localized Magnetic Perturbations

Authors: P. Xie, Y. Sun, M. Jia, A. Loarte, Y. Q. Liu, C. Ye, S. Gu, H. Sheng, Y. Liang, Q. Ma, H. Yang, C. A. Paz-Soldan, G. Deng, S. Fu, G. Chen, K. He, T. Jia, D. Lu, B. Lv, J. Qian, H. H. Wang, S. Wang, D. Weisberg, X. Wu, W. Xu , et al. (9 additional authors not shown)

Abstract: We report the suppression of Type-I Edge Localized Modes (ELMs) in the EAST tokamak under ITER baseline conditions using $n = 4$ Resonant Magnetic Perturbations (RMPs), while maintaining energy confinement. Achieving RMP-ELM suppression requires a normalized plasma beta ($β_N$) exceeding 1.8 in a target plasma with $q_{95}\approx 3.1$ and tungsten divertors. Quasi-linear modeling shows high plasma… ▽ More We report the suppression of Type-I Edge Localized Modes (ELMs) in the EAST tokamak under ITER baseline conditions using $n = 4$ Resonant Magnetic Perturbations (RMPs), while maintaining energy confinement. Achieving RMP-ELM suppression requires a normalized plasma beta ($β_N$) exceeding 1.8 in a target plasma with $q_{95}\approx 3.1$ and tungsten divertors. Quasi-linear modeling shows high plasma beta enhances RMP-driven neoclassical toroidal viscosity torque, reducing field penetration thresholds. These findings demonstrate the feasibility and efficiency of high $n$ RMPs for ELM suppression in ITER. △ Less

Submitted 6 August, 2024; originally announced August 2024.

Comments: 6 pages, 4 figures

Optimizing Cox Models with Stochastic Gradient Descent: Theoretical Foundations and Practical Guidances

Authors: Lang Zeng, Weijing Tang, Zhao Ren, Ying Ding

Abstract: Optimizing Cox regression and its neural network variants poses substantial computational challenges in large-scale studies. Stochastic gradient descent (SGD), known for its scalability in model optimization, has recently been adapted to optimize Cox models. Unlike its conventional application, which typically targets a sum of independent individual loss, SGD for Cox models updates parameters base… ▽ More Optimizing Cox regression and its neural network variants poses substantial computational challenges in large-scale studies. Stochastic gradient descent (SGD), known for its scalability in model optimization, has recently been adapted to optimize Cox models. Unlike its conventional application, which typically targets a sum of independent individual loss, SGD for Cox models updates parameters based on the partial likelihood of a subset of data. Despite its empirical success, the theoretical foundation for optimizing Cox partial likelihood with SGD is largely underexplored. In this work, we demonstrate that the SGD estimator targets an objective function that is batch-size-dependent. We establish that the SGD estimator for the Cox neural network (Cox-NN) is consistent and achieves the optimal minimax convergence rate up to a polylogarithmic factor. For Cox regression, we further prove the $\sqrt{n}$-consistency and asymptotic normality of the SGD estimator, with variance depending on the batch size. Furthermore, we quantify the impact of batch size on Cox-NN training and its effect on the SGD estimator's asymptotic efficiency in Cox regression. These findings are validated by extensive numerical experiments and provide guidance for selecting batch sizes in SGD applications. Finally, we demonstrate the effectiveness of SGD in a real-world application where GD is unfeasible due to the large scale of data. △ Less

Submitted 5 August, 2024; originally announced August 2024.

Multibeam Hybrid Transmitarray Based on Polarization Rotating Metasurface With Reconfigurable Bidirectional Radiation

Authors: Fan Qin, Yifei Liu, Chao Gu, Linfeng Zeng, Wenchi Cheng, Hailin Zhang, Steven Gao

Abstract: This paper proposes a bidirectional multibeam hybrid transmitarray (HTA) employing a transmission polarization-rotating metasurface (TPRM). A novel configuration is introduced to facilitate bidirectional beam scanning by combining the transmitarray (TA) and folded-transmitarray (FTA). To accomplish the reconfiguration of both unidirectional and bidirectional radiation states in the +z, -z, and +/-… ▽ More This paper proposes a bidirectional multibeam hybrid transmitarray (HTA) employing a transmission polarization-rotating metasurface (TPRM). A novel configuration is introduced to facilitate bidirectional beam scanning by combining the transmitarray (TA) and folded-transmitarray (FTA). To accomplish the reconfiguration of both unidirectional and bidirectional radiation states in the +z, -z, and +/-z directions, a polarization switchable multi-feed array (MFA) is placed at the focal plane between the TA and FTA, radiating x-polarization, y-polarization, and 45-degree oblique polarization waves, respectively. Meanwhile, the proposed antenna can achieve multibeam radiation in the three aforementioned states by switching the polarization of the MFA. To demonstrate the operating principle, a prototype has been designed, simulated, and fabricated. The measured results agree well with the simulated results. The simulated and measured results indicate that the proposed design can generate reconfigurable multibeam in both forward and backward directions, either separately or simultaneously. In the unidirectional states, forward and backward beam scanning is achieved within an angular range of +/-30° and +/-22°, respectively, with peak gains of 23.6 dBi and 23.1 dBi. A simultaneous forward and backward beam scanning of +/-40° and +/-22° is achieved in the hybrid radiation state, with peak gains of 19.4 dBi and 19.3 dBi, respectively. The proposed antenna array design offers several advantages, including bidirectional low-loss beam scanning, a simple structure, low power consumption, and a low profile. △ Less

Submitted 2 August, 2024; originally announced August 2024.

Comments: 12 pages, 26 figures, published to TAP

ThinkRepair: Self-Directed Automated Program Repair

Authors: Xin Yin, Chao Ni, Shaohua Wang, Zhenhao Li, Limin Zeng, Xiaohu Yang

Abstract: Though many approaches have been proposed for Automated Program Repair (APR) and indeed achieved remarkable performance, they still have limitations in fixing bugs that require analyzing and reasoning about the logic of the buggy program. Recently, large language models (LLMs) instructed by prompt engineering have attracted much attention for their powerful ability to address many kinds of tasks i… ▽ More Though many approaches have been proposed for Automated Program Repair (APR) and indeed achieved remarkable performance, they still have limitations in fixing bugs that require analyzing and reasoning about the logic of the buggy program. Recently, large language models (LLMs) instructed by prompt engineering have attracted much attention for their powerful ability to address many kinds of tasks including bug-fixing. However, the quality of the prompt will highly affect the ability of LLMs and manually constructing high-quality prompts is a costly endeavor. To address this limitation, we propose a self-directed LLM-based automated program repair, ThinkRepair, with two main phases: collection phase and fixing phase. The former phase automatically collects various chains of thoughts that constitute pre-fixed knowledge by instructing LLMs with the Chain-of-Thought (CoT) prompt. The latter phase targets fixing a bug by first selecting examples for few-shot learning and second automatically interacting with LLMs, optionally appending with feedback of testing information. Evaluations on two widely studied datasets (Defects4J and QuixBugs) by comparing ThinkRepair with 12 SOTA APRs indicate the priority of ThinkRepair in fixing bugs. Notably, ThinkRepair fixes 98 bugs and improves baselines by 27%-344.4% on Defects4J V1.2. On Defects4J V2.0, ThinkRepair fixes 12-65 more bugs than the SOTA APRs. Additionally, ThinkRepair also makes a considerable improvement on QuixBugs (31 for Java and 21 for Python at most). △ Less

Submitted 30 July, 2024; originally announced July 2024.

Comments: Accepted By ISSTA'24

Structural damage detection via hierarchical damage information with volumetric assessment

Authors: Isaac Osei Agyemang, Jianwen Chen, Liaoyuan Zeng, Isaac Adjei-Mensah, Daniel Acheampong, Gordon Owusu Boateng, Adu Asare Baffour

Abstract: Image environments and noisy labels hinder deep learning-based inference models in structural damage detection. Post-detection, there is the challenge of reliance on manual assessments of detected damages. As a result, Guided-DetNet, characterized by Generative Attention Module (GAM), Hierarchical Elimination Algorithm (HEA), and Volumetric Contour Visual Assessment (VCVA), is proposed to mitigate… ▽ More Image environments and noisy labels hinder deep learning-based inference models in structural damage detection. Post-detection, there is the challenge of reliance on manual assessments of detected damages. As a result, Guided-DetNet, characterized by Generative Attention Module (GAM), Hierarchical Elimination Algorithm (HEA), and Volumetric Contour Visual Assessment (VCVA), is proposed to mitigate complex image environments, noisy labeling, and post-detection manual assessment of structural damages. GAM leverages cross-horizontal and cross-vertical patch merging and cross foreground-background feature fusion to generate varied features to mitigate complex image environments. HEA addresses noisy labeling using hierarchical relationships among classes to refine instances given an image by eliminating unlikely class categories. VCVA assesses the severity of detected damages via volumetric representation and quantification leveraging the Dirac delta distribution. A comprehensive quantitative study, two robustness tests, and an application scenario based on the PEER Hub Image-Net dataset substantiate Guided-DetNet's promising performances. Guided-DetNet outperformed the best-compared models in a triple classification task by a difference of not less than 3% and not less than 2% in a dual detection task under varying metrics. △ Less

Submitted 29 July, 2024; originally announced July 2024.

LKCell: Efficient Cell Nuclei Instance Segmentation with Large Convolution Kernels

Authors: Ziwei Cui, Jingfeng Yao, Lunbin Zeng, Juan Yang, Wenyu Liu, Xinggang Wang

Abstract: The segmentation of cell nuclei in tissue images stained with the blood dye hematoxylin and eosin (H$\&$E) is essential for various clinical applications and analyses. Due to the complex characteristics of cellular morphology, a large receptive field is considered crucial for generating high-quality segmentation. However, previous methods face challenges in achieving a balance between the receptiv… ▽ More The segmentation of cell nuclei in tissue images stained with the blood dye hematoxylin and eosin (H$\&$E) is essential for various clinical applications and analyses. Due to the complex characteristics of cellular morphology, a large receptive field is considered crucial for generating high-quality segmentation. However, previous methods face challenges in achieving a balance between the receptive field and computational burden. To address this issue, we propose LKCell, a high-accuracy and efficient cell segmentation method. Its core insight lies in unleashing the potential of large convolution kernels to achieve computationally efficient large receptive fields. Specifically, (1) We transfer pre-trained large convolution kernel models to the medical domain for the first time, demonstrating their effectiveness in cell segmentation. (2) We analyze the redundancy of previous methods and design a new segmentation decoder based on large convolution kernels. It achieves higher performance while significantly reducing the number of parameters. We evaluate our method on the most challenging benchmark and achieve state-of-the-art results (0.5080 mPQ) in cell nuclei instance segmentation with only 21.6% FLOPs compared with the previous leading method. Our source code and models are available at https://github.com/hustvl/LKCell. △ Less

Submitted 25 July, 2024; originally announced July 2024.

Causal estimands and identification of time-varying effects in non-stationary time series from N-of-1 mobile device data

Authors: Xiaoxuan Cai, Li Zeng, Charlotte Fowler, Lisa Dixon, Dost Ongur, Justin T. Baker, Jukka-Pekka Onnela, Linda Valeri

Abstract: Mobile technology (mobile phones and wearable devices) generates continuous data streams encompassing outcomes, exposures and covariates, presented as intensive longitudinal or multivariate time series data. The high frequency of measurements enables granular and dynamic evaluation of treatment effect, revealing their persistence and accumulation over time. Existing methods predominantly focus on… ▽ More Mobile technology (mobile phones and wearable devices) generates continuous data streams encompassing outcomes, exposures and covariates, presented as intensive longitudinal or multivariate time series data. The high frequency of measurements enables granular and dynamic evaluation of treatment effect, revealing their persistence and accumulation over time. Existing methods predominantly focus on the contemporaneous effect, temporal-average, or population-average effects, assuming stationarity or invariance of treatment effects over time, which are inadequate both conceptually and statistically to capture dynamic treatment effects in personalized mobile health data. We here propose new causal estimands for multivariate time series in N-of-1 studies. These estimands summarize how time-varying exposures impact outcomes in both short- and long-term. We propose identifiability assumptions and a g-formula estimator that accounts for exposure-outcome and outcome-covariate feedback. The g-formula employs a state space model framework innovatively to accommodate time-varying behavior of treatment effects in non-stationary time series. We apply the proposed method to a multi-year smartphone observational study of bipolar patients and estimate the dynamic effect of phone-based communication on mood of patients with bipolar disorder in an N-of-1 setting. Our approach reveals substantial heterogeneity in treatment effects over time and across individuals. A simulation-based strategy is also proposed for the development of a short-term, dynamic, and personalized treatment recommendation based on patient's past information, in combination with a novel positivity diagnostics plot, validating proper causal inference in time series data. △ Less

Submitted 24 July, 2024; originally announced July 2024.

The finite basis problem for additively idempotent semirings of order four, I

Authors: Miaomiao Ren, Junyang Liu, Lingli Zeng, Menglong Chen

Abstract: We study the finite basis problem for 4-element additively idempotent semirings whose additive reducts are semilattices of height 1. Up to isomorphism, there are 58 such algebras. We show that 49 of them are finitely based and the remaining ones are nonfinitely based. We study the finite basis problem for 4-element additively idempotent semirings whose additive reducts are semilattices of height 1. Up to isomorphism, there are 58 such algebras. We show that 49 of them are finitely based and the remaining ones are nonfinitely based. △ Less

Submitted 21 July, 2024; originally announced July 2024.

Comments: 30pages

MSC Class: 16Y60; 03C05; 08B05

Edge Graph Intelligence: Reciprocally Empowering Edge Networks with Graph Intelligence

Authors: Liekang Zeng, Shengyuan Ye, Xu Chen, Xiaoxi Zhang, Ju Ren, Jian Tang, Yang Yang, Xuemin, Shen

Abstract: Recent years have witnessed a thriving growth of computing facilities connected at the network edge, cultivating edge computing networks as a fundamental infrastructure for supporting miscellaneous intelligent services. Meanwhile, Artificial Intelligence frontiers have extrapolated Machine Learning to the graph domain and promoted Graph Intelligence (GI), which unlocks unprecedented ability in lea… ▽ More Recent years have witnessed a thriving growth of computing facilities connected at the network edge, cultivating edge computing networks as a fundamental infrastructure for supporting miscellaneous intelligent services. Meanwhile, Artificial Intelligence frontiers have extrapolated Machine Learning to the graph domain and promoted Graph Intelligence (GI), which unlocks unprecedented ability in learning from massive data in graph structures. Given the inherent relation between graphs and networks, the interdiscipline of graph representation learning and edge networks, i.e., Edge GI or EGI, has revealed a novel interplay between them -- GI models principally open a new door for modeling, understanding, and optimizing edge networks, and conversely, edge networks serve as physical support for training, deploying, and accelerating GI models. Driven by this delicate closed-loop, EGI can be widely recognized as a promising solution to fully unleash the potential of edge computing power and is garnering significant attention. Nevertheless, research on EGI yet remains nascent, and there is a soaring demand within both the communications and AI communities for a dedicated venue to share recent advancements. To this end, this paper promotes the concept of EGI, explores its scope and core principles, and conducts a comprehensive survey concerning recent research efforts on this emerging field and specifically, introduces and discusses: 1) fundamentals of edge computing and graph representation learning, 2) emerging techniques centering on the closed loop between graph intelligence and edge networks, and 3) open challenges and research opportunities of future EGI. By bridging the gap across communication, networking, and graph learning areas, we believe that this survey can garner increased attention, foster meaningful discussions, and inspire further research ideas in EGI. △ Less

Submitted 7 July, 2024; originally announced July 2024.

Comments: 38 pages, 14 figures

Automated Peer Reviewing in Paper SEA: Standardization, Evaluation, and Analysis

Authors: Jianxiang Yu, Zichen Ding, Jiaqi Tan, Kangyang Luo, Zhenmin Weng, Chenghua Gong, Long Zeng, Renjing Cui, Chengcheng Han, Qiushi Sun, Zhiyong Wu, Yunshi Lan, Xiang Li

Abstract: In recent years, the rapid increase in scientific papers has overwhelmed traditional review mechanisms, resulting in varying quality of publications. Although existing methods have explored the capabilities of Large Language Models (LLMs) for automated scientific reviewing, their generated contents are often generic or partial. To address the issues above, we introduce an automated paper reviewing… ▽ More In recent years, the rapid increase in scientific papers has overwhelmed traditional review mechanisms, resulting in varying quality of publications. Although existing methods have explored the capabilities of Large Language Models (LLMs) for automated scientific reviewing, their generated contents are often generic or partial. To address the issues above, we introduce an automated paper reviewing framework SEA. It comprises of three modules: Standardization, Evaluation, and Analysis, which are represented by models SEA-S, SEA-E, and SEA-A, respectively. Initially, SEA-S distills data standardization capabilities of GPT-4 for integrating multiple reviews for a paper. Then, SEA-E utilizes standardized data for fine-tuning, enabling it to generate constructive reviews. Finally, SEA-A introduces a new evaluation metric called mismatch score to assess the consistency between paper contents and reviews. Moreover, we design a self-correction strategy to enhance the consistency. Extensive experimental results on datasets collected from eight venues show that SEA can generate valuable insights for authors to improve their papers. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Numerical Analysis on the Spatiotemporal Characteristics of the Portevin-Le Chatelier Effect in Ti-12Mo Alloy

Authors: Shiyuan Luo, Yongxin Jiang, Sandrine Thuillier, Philippe Castany, Liangcai Zeng

Abstract: A simplified 3D FE model based on McCormick's model is developed to numerically predict the spatiotemporal behaviors of the PLC effect in Ti-12Mo alloy tensile tests at 350 degrees C with strain rates from the order of $10^{-4}$ s$^{-1}$ to $10^{-2}$ s$^{-1}$. The material parameter identification procedure is firstly presented in details, and the simulated results are highly consistent with exper… ▽ More A simplified 3D FE model based on McCormick's model is developed to numerically predict the spatiotemporal behaviors of the PLC effect in Ti-12Mo alloy tensile tests at 350 degrees C with strain rates from the order of $10^{-4}$ s$^{-1}$ to $10^{-2}$ s$^{-1}$. The material parameter identification procedure is firstly presented in details, and the simulated results are highly consistent with experimental ones, especially in terms of stress drop magnitudes and PLC band widths. The distribution of simulated stress drop magnitudes at a constant tensile velocity (0.01 mm/s) follows a normal distribution and its peak value is in the range of 26-28 MPa. Furthermore, the simulated band width slightly fluctuates with the increase of true strain and its average value is about 1.5 mm. Besides, the staircase behavior of strain-time curves and the hopping propagation of the PLC band are observed in Ti-12Mo alloy tensile process, which are related to the strain localization and stress drop magnitudes. △ Less

Submitted 12 July, 2024; originally announced July 2024.

Journal ref: Metals and Materials International, 2023, 29 (2), pp.269-279

Skywork-Math: Data Scaling Laws for Mathematical Reasoning in Large Language Models -- The Story Goes On

Authors: Liang Zeng, Liangjun Zhong, Liang Zhao, Tianwen Wei, Liu Yang, Jujie He, Cheng Cheng, Rui Hu, Yang Liu, Shuicheng Yan, Han Fang, Yahui Zhou

Abstract: In this paper, we investigate the underlying factors that potentially enhance the mathematical reasoning capabilities of large language models (LLMs). We argue that the data scaling law for math reasoning capabilities in modern LLMs is far from being saturated, highlighting how the model's quality improves with increases in data quantity. To support this claim, we introduce the Skywork-Math model… ▽ More In this paper, we investigate the underlying factors that potentially enhance the mathematical reasoning capabilities of large language models (LLMs). We argue that the data scaling law for math reasoning capabilities in modern LLMs is far from being saturated, highlighting how the model's quality improves with increases in data quantity. To support this claim, we introduce the Skywork-Math model series, supervised fine-tuned (SFT) on common 7B LLMs using our proposed 2.5M-instance Skywork-MathQA dataset. Skywork-Math 7B has achieved impressive accuracies of 51.2% on the competition-level MATH benchmark and 83.9% on the GSM8K benchmark using only SFT data, outperforming an early version of GPT-4 on MATH. The superior performance of Skywork-Math models contributes to our novel two-stage data synthesis and model SFT pipelines, which include three different augmentation methods and a diverse seed problem set, ensuring both the quantity and quality of Skywork-MathQA dataset across varying difficulty levels. Most importantly, we provide several practical takeaways to enhance math reasoning abilities in LLMs for both research and industry applications. △ Less

Submitted 17 July, 2024; v1 submitted 11 July, 2024; originally announced July 2024.

Mobile Robot Oriented Large-Scale Indoor Dataset for Dynamic Scene Understanding

Authors: Yifan Tang, Cong Tai, Fangxing Chen, Wanting Zhang, Tao Zhang, Xueping Liu, Yongjin Liu, Long Zeng

Abstract: Most existing robotic datasets capture static scene data and thus are limited in evaluating robots' dynamic performance. To address this, we present a mobile robot oriented large-scale indoor dataset, denoted as THUD (Tsinghua University Dynamic) robotic dataset, for training and evaluating their dynamic scene understanding algorithms. Specifically, the THUD dataset construction is first detailed,… ▽ More Most existing robotic datasets capture static scene data and thus are limited in evaluating robots' dynamic performance. To address this, we present a mobile robot oriented large-scale indoor dataset, denoted as THUD (Tsinghua University Dynamic) robotic dataset, for training and evaluating their dynamic scene understanding algorithms. Specifically, the THUD dataset construction is first detailed, including organization, acquisition, and annotation methods. It comprises both real-world and synthetic data, collected with a real robot platform and a physical simulation platform, respectively. Our current dataset includes 13 larges-scale dynamic scenarios, 90K image frames, 20M 2D/3D bounding boxes of static and dynamic objects, camera poses, and IMU. The dataset is still continuously expanding. Then, the performance of mainstream indoor scene understanding tasks, e.g. 3D object detection, semantic segmentation, and robot relocalization, is evaluated on our THUD dataset. These experiments reveal serious challenges for some robot scene understanding tasks in dynamic scenes. By sharing this dataset, we aim to foster and iterate new mobile robot algorithms quickly for robot actual working dynamic environment, i.e. complex crowded dynamic scenes. △ Less

Submitted 30 June, 2024; v1 submitted 28 June, 2024; originally announced June 2024.

Comments: This version has been accepted by ICRA2024 and the dataset has been published, where the link can be found in the paper

Journal ref: IEEE International Conference on Robotics & Automation,2024

Online Optimization of DNN Inference Network Utility in Collaborative Edge Computing

Authors: Rui Li, Tao Ouyang, Liekang Zeng, Guocheng Liao, Zhi Zhou, Xu Chen

Abstract: Collaborative Edge Computing (CEC) is an emerging paradigm that collaborates heterogeneous edge devices as a resource pool to compute DNN inference tasks in proximity such as edge video analytics. Nevertheless, as the key knob to improve network utility in CEC, existing works mainly focus on the workload routing strategies among edge devices with the aim of minimizing the routing cost, remaining a… ▽ More Collaborative Edge Computing (CEC) is an emerging paradigm that collaborates heterogeneous edge devices as a resource pool to compute DNN inference tasks in proximity such as edge video analytics. Nevertheless, as the key knob to improve network utility in CEC, existing works mainly focus on the workload routing strategies among edge devices with the aim of minimizing the routing cost, remaining an open question for joint workload allocation and routing optimization problem from a system perspective. To this end, this paper presents a holistic, learned optimization for CEC towards maximizing the total network utility in an online manner, even though the utility functions of task input rates are unknown a priori. In particular, we characterize the CEC system in a flow model and formulate an online learning problem in a form of cross-layer optimization. We propose a nested-loop algorithm to solve workload allocation and distributed routing iteratively, using the tools of gradient sampling and online mirror descent. To improve the convergence rate over the nested-loop version, we further devise a single-loop algorithm. Rigorous analysis is provided to show its inherent convexity, efficient convergence, as well as algorithmic optimality. Finally, extensive numerical simulations demonstrate the superior performance of our solutions. △ Less

Submitted 27 June, 2024; originally announced June 2024.

Comments: Accepted by IEEE/ACM TRANSACTIONS ON NETWORKING (ToN)

Upgrading the Submillimeter Array: wSMA and beyond

Authors: Paul K. Grimes, Garrett K. Keating, Raymond Blundell, Robert D. Christensen, Mark Gurwell, Attila Kovacs, Timothy Norton, Scott N. Paine, Ramprasad Rao, Edward C. -Y. Tong, Jonathan Weintroub, David Wilner, Robert W. Wilson, Lingzhen Zeng, Qizhou Zhang

Abstract: The Submillimeter Array (SMA) is an array of 8 antennas operating at millimeter and submillimeter wavelengths on Maunakea, Hawaii, operated by the Smithsonian Astrophysical Observatory and Academia Sinica Institute of Astronomy and Astrophysics, Taiwan. Over the past several years, we have been preparing a major upgrade to the SMA that will replace the aging original receiver cryostats and receive… ▽ More The Submillimeter Array (SMA) is an array of 8 antennas operating at millimeter and submillimeter wavelengths on Maunakea, Hawaii, operated by the Smithsonian Astrophysical Observatory and Academia Sinica Institute of Astronomy and Astrophysics, Taiwan. Over the past several years, we have been preparing a major upgrade to the SMA that will replace the aging original receiver cryostats and receiver cartridges with all new cryostats and new 230 and 345 GHz receiver designs. This wideband upgrade (wSMA) will also include significantly increased instantaneous bandwidth, improved sensitivity, and greater capabilities for dual frequency observations. In this paper, we will describe the wSMA receiver upgrade and status, as well as the future upgrades that will be enabled by the deployment of the wSMA receivers. △ Less

Submitted 24 June, 2024; originally announced June 2024.

Comments: To be published in the proceedings of SPIE Astronomical Telescopes + Instrumentation 2024, paper number 13096-122

Synthesis of Electron Microbunching Rotation for Generating Isolated Attosecond Soft X-ray Free-electron Laser Pulses

Authors: Hao Sun, Xiaofan Wang, Li Zeng, Weiqing Zhang

Abstract: Attosecond x-ray pulses play a crucial role in the study of ultrafast phenomena occurring within inner and valence electrons. Especially isolated attosecond pulses with high photon energy and high peak power are of great significance in single-shot imaging in the soft x-ray region, life sciences, and attosecond pump-probe experiments. In modern accelerators, laser manipulation of electrons can be… ▽ More Attosecond x-ray pulses play a crucial role in the study of ultrafast phenomena occurring within inner and valence electrons. Especially isolated attosecond pulses with high photon energy and high peak power are of great significance in single-shot imaging in the soft x-ray region, life sciences, and attosecond pump-probe experiments. In modern accelerators, laser manipulation of electrons can be used to tailor the ultrafast properties of free-electron laser (FEL) pulses. In this paper, we propose a novel laser manipulation technique that makes use of two laser beams with mutual delays and tilted wavefronts to synthesize microbunching rotation on the scale of infrared laser wavelengths within the electron bunch for generating isolated attosecond soft x-ray pulses. This microbunching rotation ultimately leads to an enhanced current contrast ratio between the main peak and the surrounding satellite peaks within the bunch. By properly accounting for the longitudinal space charge fields within the FEL undulator, a tapered undulator can further suppress the side peaks in the radiation pulse and enable the selection of an isolated, hundred-attosecond, GW-level soft x-ray pulse. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Q*: Improving Multi-step Reasoning for LLMs with Deliberative Planning

Authors: Chaojie Wang, Yanchen Deng, Zhiyi Lyu, Liang Zeng, Jujie He, Shuicheng Yan, Bo An

Abstract: Large Language Models (LLMs) have demonstrated impressive capability in many natural language tasks. However, the auto-regressive generation process makes LLMs prone to produce errors, hallucinations and inconsistent statements when performing multi-step reasoning. In this paper, by casting multi-step reasoning of LLMs as a heuristic search problem, we aim to alleviate the pathology by introducing… ▽ More Large Language Models (LLMs) have demonstrated impressive capability in many natural language tasks. However, the auto-regressive generation process makes LLMs prone to produce errors, hallucinations and inconsistent statements when performing multi-step reasoning. In this paper, by casting multi-step reasoning of LLMs as a heuristic search problem, we aim to alleviate the pathology by introducing Q*, a general, versatile and agile framework for guiding LLMs decoding process with deliberative planning. By learning a plug-and-play Q-value model as heuristic function for estimating expected future rewards, our Q* can effectively guide LLMs to select the most promising next reasoning step without fine-tuning LLMs for the current task, which avoids the significant computational overhead and potential risk of performance degeneration on other tasks. Extensive experiments on GSM8K, MATH and MBPP demonstrate the superiority of our method, contributing to improving the reasoning performance of existing open-source LLMs. △ Less

Submitted 22 July, 2024; v1 submitted 20 June, 2024; originally announced June 2024.

Magnetic field in mini starburst complex Sgr B2

Authors: Xing Pan, Qizhou Zhang, Keping Qiu, Ramprasad Rao, Lingzhen Zeng, Xing Lu, Junhao Liu

Abstract: We report the first arcsecond-resolution observations of the magnetic field in the mini starburst complex Sgr B2. SMA polarization observations revealed magnetic field morphology in three dense cores of Sgr B2 N(orth), M(ain), and S(outh). The total plane-of-sky magnetic field strengths in these cores are estimated to be 4.3-10.0 mG, 6.2-14.7 mG, and 1.9-4.5 mG derived from the angular dispersion… ▽ More We report the first arcsecond-resolution observations of the magnetic field in the mini starburst complex Sgr B2. SMA polarization observations revealed magnetic field morphology in three dense cores of Sgr B2 N(orth), M(ain), and S(outh). The total plane-of-sky magnetic field strengths in these cores are estimated to be 4.3-10.0 mG, 6.2-14.7 mG, and 1.9-4.5 mG derived from the angular dispersion function method after applying the correction factors of 0.21 and 0.5. Combining with analyses of the parsec-scale polarization data from SOFIA, we found that a magnetically supercritical condition is present from the cloud-scale ($\sim$10 pc) to core-scale ($\sim$0.2 pc) in Sgr B2, which is consistent with the burst of star formation activities in the region likely resulted from a multi-scale gravitational collapse from the cloud to dense cores. △ Less

Submitted 18 June, 2024; v1 submitted 17 June, 2024; originally announced June 2024.

Comments: 17 pages, 4 figures, accepted for publication in ApJ

EgoExo-Fitness: Towards Egocentric and Exocentric Full-Body Action Understanding

Authors: Yuan-Ming Li, Wei-Jin Huang, An-Lan Wang, Ling-An Zeng, Jing-Ke Meng, Wei-Shi Zheng

Abstract: We present EgoExo-Fitness, a new full-body action understanding dataset, featuring fitness sequence videos recorded from synchronized egocentric and fixed exocentric (third-person) cameras. Compared with existing full-body action understanding datasets, EgoExo-Fitness not only contains videos from first-person perspectives, but also provides rich annotations. Specifically, two-level temporal bound… ▽ More We present EgoExo-Fitness, a new full-body action understanding dataset, featuring fitness sequence videos recorded from synchronized egocentric and fixed exocentric (third-person) cameras. Compared with existing full-body action understanding datasets, EgoExo-Fitness not only contains videos from first-person perspectives, but also provides rich annotations. Specifically, two-level temporal boundaries are provided to localize single action videos along with sub-steps of each action. More importantly, EgoExo-Fitness introduces innovative annotations for interpretable action judgement--including technical keypoint verification, natural language comments on action execution, and action quality scores. Combining all of these, EgoExo-Fitness provides new resources to study egocentric and exocentric full-body action understanding across dimensions of "what", "when", and "how well". To facilitate research on egocentric and exocentric full-body action understanding, we construct benchmarks on a suite of tasks (i.e., action classification, action localization, cross-view sequence verification, cross-view skill determination, and a newly proposed task of guidance-based execution verification), together with detailed analysis. Code and data will be available at https://github.com/iSEE-Laboratory/EgoExo-Fitness/tree/main. △ Less

Submitted 16 July, 2024; v1 submitted 13 June, 2024; originally announced June 2024.

Comments: Accepted by ECCV2024

Skywork-MoE: A Deep Dive into Training Techniques for Mixture-of-Experts Language Models

Authors: Tianwen Wei, Bo Zhu, Liang Zhao, Cheng Cheng, Biye Li, Weiwei Lü, Peng Cheng, Jianhao Zhang, Xiaoyu Zhang, Liang Zeng, Xiaokun Wang, Yutuan Ma, Rui Hu, Shuicheng Yan, Han Fang, Yahui Zhou

Abstract: In this technical report, we introduce the training methodologies implemented in the development of Skywork-MoE, a high-performance mixture-of-experts (MoE) large language model (LLM) with 146 billion parameters and 16 experts. It is initialized from the pre-existing dense checkpoints of our Skywork-13B model. We explore the comparative effectiveness of upcycling versus training from scratch initi… ▽ More In this technical report, we introduce the training methodologies implemented in the development of Skywork-MoE, a high-performance mixture-of-experts (MoE) large language model (LLM) with 146 billion parameters and 16 experts. It is initialized from the pre-existing dense checkpoints of our Skywork-13B model. We explore the comparative effectiveness of upcycling versus training from scratch initializations. Our findings suggest that the choice between these two approaches should consider both the performance of the existing dense checkpoints and the MoE training budget. We highlight two innovative techniques: gating logit normalization, which improves expert diversification, and adaptive auxiliary loss coefficients, allowing for layer-specific adjustment of auxiliary loss coefficients. Our experimental results validate the effectiveness of these methods. Leveraging these techniques and insights, we trained our upcycled Skywork-MoE on a condensed subset of our SkyPile corpus. The evaluation results demonstrate that our model delivers strong performance across a wide range of benchmarks. △ Less

Submitted 2 June, 2024; originally announced June 2024.

Simplify Implant Depth Prediction as Video Grounding: A Texture Perceive Implant Depth Prediction Network

Authors: Xinquan Yang, Xuguang Li, Xiaoling Luo, Leilei Zeng, Yudi Zhang, Linlin Shen, Yongqiang Deng

Abstract: Surgical guide plate is an important tool for the dental implant surgery. However, the design process heavily relies on the dentist to manually simulate the implant angle and depth. When deep neural networks have been applied to assist the dentist quickly locates the implant position, most of them are not able to determine the implant depth. Inspired by the video grounding task which localizes the… ▽ More Surgical guide plate is an important tool for the dental implant surgery. However, the design process heavily relies on the dentist to manually simulate the implant angle and depth. When deep neural networks have been applied to assist the dentist quickly locates the implant position, most of them are not able to determine the implant depth. Inspired by the video grounding task which localizes the starting and ending time of the target video segment, in this paper, we simplify the implant depth prediction as video grounding and develop a Texture Perceive Implant Depth Prediction Network (TPNet), which enables us to directly output the implant depth without complex measurements of oral bone. TPNet consists of an implant region detector (IRD) and an implant depth prediction network (IDPNet). IRD is an object detector designed to crop the candidate implant volume from the CBCT, which greatly saves the computation resource. IDPNet takes the cropped CBCT data to predict the implant depth. A Texture Perceive Loss (TPL) is devised to enable the encoder of IDPNet to perceive the texture variation among slices. Extensive experiments on a large dental implant dataset demonstrated that the proposed TPNet achieves superior performance than the existing methods. △ Less

Submitted 6 June, 2024; originally announced June 2024.

Journal ref: MICCAI'2024

LongSkywork: A Training Recipe for Efficiently Extending Context Length in Large Language Models

Authors: Liang Zhao, Tianwen Wei, Liang Zeng, Cheng Cheng, Liu Yang, Peng Cheng, Lijie Wang, Chenxia Li, Xuejie Wu, Bo Zhu, Yimeng Gan, Rui Hu, Shuicheng Yan, Han Fang, Yahui Zhou

Abstract: We introduce LongSkywork, a long-context Large Language Model (LLM) capable of processing up to 200,000 tokens. We provide a training recipe for efficiently extending context length of LLMs. We identify that the critical element in enhancing long-context processing capability is to incorporate a long-context SFT stage following the standard SFT stage. A mere 200 iterations can convert the standard… ▽ More We introduce LongSkywork, a long-context Large Language Model (LLM) capable of processing up to 200,000 tokens. We provide a training recipe for efficiently extending context length of LLMs. We identify that the critical element in enhancing long-context processing capability is to incorporate a long-context SFT stage following the standard SFT stage. A mere 200 iterations can convert the standard SFT model into a long-context model. To reduce the effort in collecting and annotating data for long-context language modeling, we develop two novel methods for creating synthetic data. These methods are applied during the continual pretraining phase as well as the Supervised Fine-Tuning (SFT) phase, greatly enhancing the training efficiency of our long-context LLMs. Our findings suggest that synthetic long-context SFT data can surpass the performance of data curated by humans to some extent. LongSkywork achieves outstanding performance on a variety of long-context benchmarks. In the Needle test, a benchmark for long-context information retrieval, our models achieved perfect accuracy across multiple context spans. Moreover, in realistic application scenarios, LongSkywork-13B demonstrates performance on par with Claude2.1, the leading long-context model, underscoring the effectiveness of our proposed methods. △ Less

Submitted 1 June, 2024; originally announced June 2024.

Expert-Token Resonance: Redefining MoE Routing through Affinity-Driven Active Selection

Authors: Jing Li, Zhijie Sun, Dachao Lin, Xuan He, Yi Lin, Binfan Zheng, Li Zeng, Rongqian Zhao, Xin Chen

Abstract: Mixture-of-Experts (MoE) architectures have emerged as a paradigm-shifting approach for large language models (LLMs), offering unprecedented computational efficiency. However, these architectures grapple with challenges of token distribution imbalance and expert homogenization, impeding optimal semantic generalization. We introduce a novel framework that redefines MoE routing through affinity-driv… ▽ More Mixture-of-Experts (MoE) architectures have emerged as a paradigm-shifting approach for large language models (LLMs), offering unprecedented computational efficiency. However, these architectures grapple with challenges of token distribution imbalance and expert homogenization, impeding optimal semantic generalization. We introduce a novel framework that redefines MoE routing through affinity-driven active selection. The innovations for the framework encompass: (1) A rigorous formulation of expert-token affinity metrics. (2) An adaptive bidirectional selection mechanism leveraging resonance between experts and tokens. (3) Theoretical derivation and experimental evidence of reduced expert capacity bounds under dynamic token distribution evolution. It is also integrated with orthogonal feature extraction module and an optimized loss function for expert localization. Our theoretical analysis demonstrates that this approach mitigates expert homogenization while enabling substantial capacity boundary reduction. Experimental validation corroborates these findings: it achieves a 40% reduction in token processed by each expert without compromising model convergence or efficacy. When coupled with communication optimizations, the training efficiency improvements of 5.4% to 46.6% can be observed. After supervised fine-tuning, it exhibits performance gains of 9.7% to 14.1% across GDAD, C-Eval, and TeleQnA benchmarks. △ Less

Submitted 30 August, 2024; v1 submitted 23 May, 2024; originally announced June 2024.

GaussianPrediction: Dynamic 3D Gaussian Prediction for Motion Extrapolation and Free View Synthesis

Authors: Boming Zhao, Yuan Li, Ziyu Sun, Lin Zeng, Yujun Shen, Rui Ma, Yinda Zhang, Hujun Bao, Zhaopeng Cui

Abstract: Forecasting future scenarios in dynamic environments is essential for intelligent decision-making and navigation, a challenge yet to be fully realized in computer vision and robotics. Traditional approaches like video prediction and novel-view synthesis either lack the ability to forecast from arbitrary viewpoints or to predict temporal dynamics. In this paper, we introduce GaussianPrediction, a n… ▽ More Forecasting future scenarios in dynamic environments is essential for intelligent decision-making and navigation, a challenge yet to be fully realized in computer vision and robotics. Traditional approaches like video prediction and novel-view synthesis either lack the ability to forecast from arbitrary viewpoints or to predict temporal dynamics. In this paper, we introduce GaussianPrediction, a novel framework that empowers 3D Gaussian representations with dynamic scene modeling and future scenario synthesis in dynamic environments. GaussianPrediction can forecast future states from any viewpoint, using video observations of dynamic scenes. To this end, we first propose a 3D Gaussian canonical space with deformation modeling to capture the appearance and geometry of dynamic scenes, and integrate the lifecycle property into Gaussians for irreversible deformations. To make the prediction feasible and efficient, a concentric motion distillation approach is developed by distilling the scene motion with key points. Finally, a Graph Convolutional Network is employed to predict the motions of key points, enabling the rendering of photorealistic images of future scenarios. Our framework shows outstanding performance on both synthetic and real-world datasets, demonstrating its efficacy in predicting and rendering future environments. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Comments: Accepted to SIGGRAPH 2024 Conference. Project Page: https://zju3dv.github.io/gaussian-prediction/

Constraining Inflation with the BICEP/Keck CMB Polarization Experiments

Authors: The BICEP/Keck Collaboration, :, P. A. R. Ade, Z. Ahmed, M. Amiri, D. Barkats, R. Basu Thakur, C. A. Bischoff, D. Beck, J. J. Bock, H. Boenish, V. Buza, J. R. Cheshire IV, J. Connors, J. Cornelison, M. Crumrine, A. Cukierman, E. V. Denison, M. Dierickx, L. Duband, M. Eiben, B. Elwood, S. Fatigoni, J. P. Filippini, M. Gao , et al. (63 additional authors not shown)

Abstract: The BICEP/$\textit{Keck}$ (BK) series of cosmic microwave background (CMB) polarization experiments has, over the past decade and a half, produced a series of field-leading constraints on cosmic inflation via measurements of the "B-mode" polarization of the CMB. Primordial B modes are directly tied to the amplitude of primordial gravitational waves (PGW), their strength parameterized by the tensor… ▽ More The BICEP/$\textit{Keck}$ (BK) series of cosmic microwave background (CMB) polarization experiments has, over the past decade and a half, produced a series of field-leading constraints on cosmic inflation via measurements of the "B-mode" polarization of the CMB. Primordial B modes are directly tied to the amplitude of primordial gravitational waves (PGW), their strength parameterized by the tensor-to-scalar ratio, $r$, and thus the energy scale of inflation. Having set the most sensitive constraints to-date on $r$, $σ(r)=0.009$ ($r_{0.05}<0.036, 95\%$ C.L.) using data through the 2018 observing season ("BK18"), the BICEP/$\textit{Keck}$ program has continued to improve its dataset in the years since. We give a brief overview of the BK program and the "BK18" result before discussing the program's ongoing efforts, including the deployment and performance of the $\textit{Keck Array}$'s successor instrument, BICEP Array, improvements to data processing and internal consistency testing, new techniques such as delensing, and how those will ultimately serve to allow BK reach $σ(r) \lesssim 0.003$ using data through the 2027 observing season. △ Less

Submitted 11 July, 2024; v1 submitted 29 May, 2024; originally announced May 2024.

Comments: 9 pages, 5 figures. Contribution to the 2024 Cosmology session of the 58th Rencontres de Moriond

FlocOff: Data Heterogeneity Resilient Federated Learning with Communication-Efficient Edge Offloading

Authors: Mulei Ma, Chenyu Gong, Liekang Zeng, Yang Yang, Liantao Wu

Abstract: Federated Learning (FL) has emerged as a fundamental learning paradigm to harness massive data scattered at geo-distributed edge devices in a privacy-preserving way. Given the heterogeneous deployment of edge devices, however, their data are usually Non-IID, introducing significant challenges to FL including degraded training accuracy, intensive communication costs, and high computing complexity.… ▽ More Federated Learning (FL) has emerged as a fundamental learning paradigm to harness massive data scattered at geo-distributed edge devices in a privacy-preserving way. Given the heterogeneous deployment of edge devices, however, their data are usually Non-IID, introducing significant challenges to FL including degraded training accuracy, intensive communication costs, and high computing complexity. Towards that, traditional approaches typically utilize adaptive mechanisms, which may suffer from scalability issues, increased computational overhead, and limited adaptability to diverse edge environments. To address that, this paper instead leverages the observation that the computation offloading involves inherent functionalities such as node matching and service correlation to achieve data reshaping and proposes Federated learning based on computing Offloading (FlocOff) framework, to address data heterogeneity and resource-constrained challenges. Specifically, FlocOff formulates the FL process with Non-IID data in edge scenarios and derives rigorous analysis on the impact of imbalanced data distribution. Based on this, FlocOff decouples the optimization in two steps, namely : (1) Minimizes the Kullback-Leibler (KL) divergence via Computation Offloading scheduling (MKL-CO); (2) Minimizes the Communication Cost through Resource Allocation (MCC-RA). Extensive experimental results demonstrate that the proposed FlocOff effectively improves model convergence and accuracy by 14.3\%-32.7\% while reducing data heterogeneity under various data distributions. △ Less

Submitted 28 May, 2024; originally announced May 2024.

QUBIQ: Uncertainty Quantification for Biomedical Image Segmentation Challenge

Authors: Hongwei Bran Li, Fernando Navarro, Ivan Ezhov, Amirhossein Bayat, Dhritiman Das, Florian Kofler, Suprosanna Shit, Diana Waldmannstetter, Johannes C. Paetzold, Xiaobin Hu, Benedikt Wiestler, Lucas Zimmer, Tamaz Amiranashvili, Chinmay Prabhakar, Christoph Berger, Jonas Weidner, Michelle Alonso-Basant, Arif Rashid, Ujjwal Baid, Wesam Adel, Deniz Ali, Bhakti Baheti, Yingbin Bai, Ishaan Bhatt, Sabri Can Cetindag , et al. (55 additional authors not shown)

Abstract: Uncertainty in medical image segmentation tasks, especially inter-rater variability, arising from differences in interpretations and annotations by various experts, presents a significant challenge in achieving consistent and reliable image segmentation. This variability not only reflects the inherent complexity and subjective nature of medical image interpretation but also directly impacts the de… ▽ More Uncertainty in medical image segmentation tasks, especially inter-rater variability, arising from differences in interpretations and annotations by various experts, presents a significant challenge in achieving consistent and reliable image segmentation. This variability not only reflects the inherent complexity and subjective nature of medical image interpretation but also directly impacts the development and evaluation of automated segmentation algorithms. Accurately modeling and quantifying this variability is essential for enhancing the robustness and clinical applicability of these algorithms. We report the set-up and summarize the benchmark results of the Quantification of Uncertainties in Biomedical Image Quantification Challenge (QUBIQ), which was organized in conjunction with International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2020 and 2021. The challenge focuses on the uncertainty quantification of medical image segmentation which considers the omnipresence of inter-rater variability in imaging datasets. The large collection of images with multi-rater annotations features various modalities such as MRI and CT; various organs such as the brain, prostate, kidney, and pancreas; and different image dimensions 2D-vs-3D. A total of 24 teams submitted different solutions to the problem, combining various baseline models, Bayesian neural networks, and ensemble model techniques. The obtained results indicate the importance of the ensemble models, as well as the need for further research to develop efficient 3D methods for uncertainty quantification methods in 3D segmentation tasks. △ Less

Submitted 24 June, 2024; v1 submitted 19 March, 2024; originally announced May 2024.

Comments: initial technical report

Galaxy: A Resource-Efficient Collaborative Edge AI System for In-situ Transformer Inference

Authors: Shengyuan Ye, Jiangsu Du, Liekang Zeng, Wenzhong Ou, Xiaowen Chu, Yutong Lu, Xu Chen

Abstract: Transformer-based models have unlocked a plethora of powerful intelligent applications at the edge, such as voice assistant in smart home. Traditional deployment approaches offload the inference workloads to the remote cloud server, which would induce substantial pressure on the backbone network as well as raise users' privacy concerns. To address that, in-situ inference has been recently recogniz… ▽ More Transformer-based models have unlocked a plethora of powerful intelligent applications at the edge, such as voice assistant in smart home. Traditional deployment approaches offload the inference workloads to the remote cloud server, which would induce substantial pressure on the backbone network as well as raise users' privacy concerns. To address that, in-situ inference has been recently recognized for edge intelligence, but it still confronts significant challenges stemming from the conflict between intensive workloads and limited on-device computing resources. In this paper, we leverage our observation that many edge environments usually comprise a rich set of accompanying trusted edge devices with idle resources and propose Galaxy, a collaborative edge AI system that breaks the resource walls across heterogeneous edge devices for efficient Transformer inference acceleration. Galaxy introduces a novel hybrid model parallelism to orchestrate collaborative inference, along with a heterogeneity-aware parallelism planning for fully exploiting the resource potential. Furthermore, Galaxy devises a tile-based fine-grained overlapping of communication and computation to mitigate the impact of tensor synchronizations on inference latency under bandwidth-constrained edge environments. Extensive evaluation based on prototype implementation demonstrates that Galaxy remarkably outperforms state-of-the-art approaches under various edge environment setups, achieving up to 2.5x end-to-end latency reduction. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Comments: Accepted by IEEE International Conference on Computer Communications 2024

On the origin of infrared bands attributed to tryptophan in Spitzer observations of IC 348

Authors: Aditya Dhariwal, Thomas H. Speak, Linshan Zeng, Amirhossein Rashidi, Brendan Moore, Olivier Berné, Anthony J. Remijan, Ilane Schroetter, Brett A. McGuire, Víctor M. Rivilla, Arnaud Belloche, Jes K. Jørgensen, Pavle Djuricanin, Takamasa Momose, Ilsa R. Cooke

Abstract: Infrared emission features toward interstellar gas of the IC 348 star cluster in Perseus have been recently proposed to originate from the amino acid tryptophan. The assignment was based on laboratory infrared spectra of tryptophan pressed into pellets, a method which is known to cause large frequency shifts compared to the gas phase. We assess the validity of the assignment based on the original… ▽ More Infrared emission features toward interstellar gas of the IC 348 star cluster in Perseus have been recently proposed to originate from the amino acid tryptophan. The assignment was based on laboratory infrared spectra of tryptophan pressed into pellets, a method which is known to cause large frequency shifts compared to the gas phase. We assess the validity of the assignment based on the original Spitzer data as well as new data from JWST. In addition, we report new spectra of tryptophan condensed in para-hydrogen matrices to compare with the observed spectra. The JWST MIRI data do not show evidence for tryptophan, despite deeper integration toward IC 348. In addition, we show that several of the lines attributed to tryptophan are likely due to instrumental artifacts. This, combined with the new laboratory data, allows us to conclude that there is no compelling evidence for the tryptophan assignment. △ Less

Submitted 26 May, 2024; originally announced May 2024.

High-yield fabrication of bubble-free magic-angle twisted bilayer graphene devices with high twist-angle homogeneity

Authors: J. Diez-Merida, I. Das, G. Di Battista, A. Diez-Carlon, M. Lee, L. Zeng, K. Watanabe, T. Taniguchi, E. Olsson, D. K. Efetov

Abstract: Magic-angle twisted bilayer graphene (MATBG) stands as one of the most versatile materials in condensed-matter physics due to its hosting of a wide variety of exotic phases while also offering convenient tunability. However, the fabrication of MATBG is still manual, and remains to be a challenging and inefficient process, with devices being highly dependent on specific fabrication methods, that of… ▽ More Magic-angle twisted bilayer graphene (MATBG) stands as one of the most versatile materials in condensed-matter physics due to its hosting of a wide variety of exotic phases while also offering convenient tunability. However, the fabrication of MATBG is still manual, and remains to be a challenging and inefficient process, with devices being highly dependent on specific fabrication methods, that often result in inconsistency and variability. In this work, we present an optimized protocol for the fabrication of MATBG samples, for which we use deterministic graphene anchoring to stabilize the twist-angle, and a careful bubble removal techniques to ensure a high twist-angle homogeneity. We use low-temperature transport experiments to extract the average twist-angle between pairs of leads. We find that up to 38 percent of the so fabricated devices show micrometer square sized regions with a twist-angle in the range 1.1 plus/minus 0.1 degrees, and a twist-angle variation of only 0.02 degrees, where in some instances such regions were up to 36 micrometer square large. We are certain that the discussed protocols can be directly transferred to non-graphene materials, and will be useful for the growing field of moire materials. △ Less

Submitted 18 May, 2024; originally announced May 2024.

NeurDB: An AI-powered Autonomous Data System

Authors: Beng Chin Ooi, Shaofeng Cai, Gang Chen, Yanyan Shen, Kian-Lee Tan, Yuncheng Wu, Xiaokui Xiao, Naili Xing, Cong Yue, Lingze Zeng, Meihui Zhang, Zhanhao Zhao

Abstract: In the wake of rapid advancements in artificial intelligence (AI), we stand on the brink of a transformative leap in data systems. The imminent fusion of AI and DB (AIxDB) promises a new generation of data systems, which will relieve the burden on end-users across all industry sectors by featuring AI-enhanced functionalities, such as personalized and automated in-database AI-powered analytics, sel… ▽ More In the wake of rapid advancements in artificial intelligence (AI), we stand on the brink of a transformative leap in data systems. The imminent fusion of AI and DB (AIxDB) promises a new generation of data systems, which will relieve the burden on end-users across all industry sectors by featuring AI-enhanced functionalities, such as personalized and automated in-database AI-powered analytics, self-driving capabilities for improved system performance, etc. In this paper, we explore the evolution of data systems with a focus on deepening the fusion of AI and DB. We present NeurDB, an AI-powered autonomous data system designed to fully embrace AI design in each major system component and provide in-database AI-powered analytics. We outline the conceptual and architectural overview of NeurDB, discuss its design choices and key components, and report its current development and future plan. △ Less

Submitted 4 July, 2024; v1 submitted 6 May, 2024; originally announced May 2024.

Powering In-Database Dynamic Model Slicing for Structured Data Analytics

Authors: Lingze Zeng, Naili Xing, Shaofeng Cai, Gang Chen, Beng Chin Ooi, Jian Pei, Yuncheng Wu

Abstract: Relational database management systems (RDBMS) are widely used for the storage and retrieval of structured data. To derive insights beyond statistical aggregation, we typically have to extract specific subdatasets from the database using conventional database operations, and then apply deep neural networks (DNN) training and inference on these respective subdatasets in a separate machine learning… ▽ More Relational database management systems (RDBMS) are widely used for the storage and retrieval of structured data. To derive insights beyond statistical aggregation, we typically have to extract specific subdatasets from the database using conventional database operations, and then apply deep neural networks (DNN) training and inference on these respective subdatasets in a separate machine learning system. The process can be prohibitively expensive, especially when there are a combinatorial number of subdatasets extracted for different analytical purposes. This calls for efficient in-database support of advanced analytical methods In this paper, we introduce LEADS, a novel SQL-aware dynamic model slicing technique to customize models for subdatasets specified by SQL queries. LEADS improves the predictive modeling of structured data via the mixture of experts (MoE) technique and maintains inference efficiency by a SQL-aware gating network. At the core of LEADS is the construction of a general model with multiple expert sub-models via MoE trained over the entire database. This SQL-aware MoE technique scales up the modeling capacity, enhances effectiveness, and preserves efficiency by activating only necessary experts via the gating network during inference. Additionally, we introduce two regularization terms during the training process of LEADS to strike a balance between effectiveness and efficiency. We also design and build an in-database inference system, called INDICES, to support end-to-end advanced structured data analytics by non-intrusively incorporating LEADS onto PostgreSQL. Our extensive experiments on real-world datasets demonstrate that LEADS consistently outperforms baseline models, and INDICES delivers effective in-database analytics with a considerable reduction in inference latency compared to traditional solutions. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Implementation of Big AI Models for Wireless Networks with Collaborative Edge Computing

Authors: Liekang Zeng, Shengyuan Ye, Xu Chen, Yang Yang

Abstract: Big Artificial Intelligence (AI) models have emerged as a crucial element in various intelligent applications at the edge, such as voice assistants in smart homes and autonomous robotics in smart factories. Training big AI models, e.g., for personalized fine-tuning and continual model refinement, poses significant challenges to edge devices due to the inherent conflict between limited computing re… ▽ More Big Artificial Intelligence (AI) models have emerged as a crucial element in various intelligent applications at the edge, such as voice assistants in smart homes and autonomous robotics in smart factories. Training big AI models, e.g., for personalized fine-tuning and continual model refinement, poses significant challenges to edge devices due to the inherent conflict between limited computing resources and intensive workload associated with training. Despite the constraints of on-device training, traditional approaches usually resort to aggregating training data and sending it to a remote cloud for centralized training. Nevertheless, this approach is neither sustainable, which strains long-range backhaul transmission and energy-consuming datacenters, nor safely private, which shares users' raw data with remote infrastructures. To address these challenges, we alternatively observe that prevalent edge environments usually contain a diverse collection of trusted edge devices with untapped idle resources, which can be leveraged for edge training acceleration. Motivated by this, in this article, we propose collaborative edge training, a novel training mechanism that orchestrates a group of trusted edge devices as a resource pool for expedited, sustainable big AI model training at the edge. As an initial step, we present a comprehensive framework for building collaborative edge training systems and analyze in-depth its merits and sustainable scheduling choices following its workflow. To further investigate the impact of its parallelism design, we empirically study a case of four typical parallelisms from the perspective of energy demand with realistic testbeds. Finally, we discuss open challenges for sustainable collaborative edge training to point to future directions of edge-centric big AI model training. △ Less

Submitted 26 April, 2024; originally announced April 2024.

A sustainable development perspective on urban-scale roof greening priorities and benefits

Authors: Jie Shao, Wei Yao, Lei Luo, Linzhou Zeng, Zhiyi He, Puzuo Wang, Huadong Guo

Abstract: Greenspaces are tightly linked to human well-being. Yet, rapid urbanization has exacerbated greenspace exposure inequality and declining human life quality. Roof greening has been recognized as an effective strategy to mitigate these negative impacts. Understanding priorities and benefits is crucial to promoting green roofs. Here, using geospatial big data, we conduct an urban-scale assessment of… ▽ More Greenspaces are tightly linked to human well-being. Yet, rapid urbanization has exacerbated greenspace exposure inequality and declining human life quality. Roof greening has been recognized as an effective strategy to mitigate these negative impacts. Understanding priorities and benefits is crucial to promoting green roofs. Here, using geospatial big data, we conduct an urban-scale assessment of roof greening at a single building level in Hong Kong from a sustainable development perspective. We identify that 85.3\% of buildings reveal potential and urgent demand for roof greening. We further find green roofs could increase greenspace exposure by \textasciitilde61\% and produce hundreds of millions (HK\$) in economic benefits annually but play a small role in urban heat mitigation (\textasciitilde0.15\degree{C}) and annual carbon emission offsets (\textasciitilde0.8\%). Our study offers a comprehensive assessment of roof greening, which could provide reference for sustainable development in cities worldwide, from data utilization to solutions and findings. △ Less

Submitted 21 April, 2024; originally announced April 2024.

Exploring the Premelting Transition through Molecular Simulations Powered by Neural Network Potentials

Authors: Limin Zeng, Ang Gao

Abstract: The system has addressed the error of "Bad character(s) in field Abstract" for no reason. Please refer to manuscript for the full abstract. The system has addressed the error of "Bad character(s) in field Abstract" for no reason. Please refer to manuscript for the full abstract. △ Less

Submitted 18 April, 2024; originally announced April 2024.

Comments: 10 pages, 6 figures

EFSA: Towards Event-Level Financial Sentiment Analysis

Authors: Tianyu Chen, Yiming Zhang, Guoxin Yu, Dapeng Zhang, Li Zeng, Qing He, Xiang Ao

Abstract: In this paper, we extend financial sentiment analysis~(FSA) to event-level since events usually serve as the subject of the sentiment in financial text. Though extracting events from the financial text may be conducive to accurate sentiment predictions, it has specialized challenges due to the lengthy and discontinuity of events in a financial text. To this end, we reconceptualize the event extrac… ▽ More In this paper, we extend financial sentiment analysis~(FSA) to event-level since events usually serve as the subject of the sentiment in financial text. Though extracting events from the financial text may be conducive to accurate sentiment predictions, it has specialized challenges due to the lengthy and discontinuity of events in a financial text. To this end, we reconceptualize the event extraction as a classification task by designing a categorization comprising coarse-grained and fine-grained event categories. Under this setting, we formulate the \textbf{E}vent-Level \textbf{F}inancial \textbf{S}entiment \textbf{A}nalysis~(\textbf{EFSA} for short) task that outputs quintuples consisting of (company, industry, coarse-grained event, fine-grained event, sentiment) from financial text. A large-scale Chinese dataset containing $12,160$ news articles and $13,725$ quintuples is publicized as a brand new testbed for our task. A four-hop Chain-of-Thought LLM-based approach is devised for this task. Systematically investigations are conducted on our dataset, and the empirical results demonstrate the benchmarking scores of existing methods and our proposed method can reach the current state-of-the-art. Our dataset and framework implementation are available at https://anonymous.4open.science/r/EFSA-645E △ Less

Submitted 8 April, 2024; originally announced April 2024.

Transform then Explore: a Simple and Effective Technique for Exploratory Combinatorial Optimization with Reinforcement Learning

Authors: Tianle Pu, Changjun Fan, Mutian Shen, Yizhou Lu, Li Zeng, Zohar Nussinov, Chao Chen, Zhong Liu

Abstract: Many complex problems encountered in both production and daily life can be conceptualized as combinatorial optimization problems (COPs) over graphs. Recent years, reinforcement learning (RL) based models have emerged as a promising direction, which treat the COPs solving as a heuristic learning problem. However, current finite-horizon-MDP based RL models have inherent limitations. They are not all… ▽ More Many complex problems encountered in both production and daily life can be conceptualized as combinatorial optimization problems (COPs) over graphs. Recent years, reinforcement learning (RL) based models have emerged as a promising direction, which treat the COPs solving as a heuristic learning problem. However, current finite-horizon-MDP based RL models have inherent limitations. They are not allowed to explore adquately for improving solutions at test time, which may be necessary given the complexity of NP-hard optimization tasks. Some recent attempts solve this issue by focusing on reward design and state feature engineering, which are tedious and ad-hoc. In this work, we instead propose a much simpler but more effective technique, named gauge transformation (GT). The technique is originated from physics, but is very effective in enabling RL agents to explore to continuously improve the solutions during test. Morever, GT is very simple, which can be implemented with less than 10 lines of Python codes, and can be applied to a vast majority of RL models. Experimentally, we show that traditional RL models with GT technique produce the state-of-the-art performances on the MaxCut problem. Furthermore, since GT is independent of any RL models, it can be seamlessly integrated into various RL frameworks, paving the way of these models for more effective explorations in the solving of general COPs. △ Less

Submitted 6 April, 2024; originally announced April 2024.

Satellite Federated Edge Learning: Architecture Design and Convergence Analysis

Authors: Yuanming Shi, Li Zeng, Jingyang Zhu, Yong Zhou, Chunxiao Jiang, Khaled B. Letaief

Abstract: The proliferation of low-earth-orbit (LEO) satellite networks leads to the generation of vast volumes of remote sensing data which is traditionally transferred to the ground server for centralized processing, raising privacy and bandwidth concerns. Federated edge learning (FEEL), as a distributed machine learning approach, has the potential to address these challenges by sharing only model paramet… ▽ More The proliferation of low-earth-orbit (LEO) satellite networks leads to the generation of vast volumes of remote sensing data which is traditionally transferred to the ground server for centralized processing, raising privacy and bandwidth concerns. Federated edge learning (FEEL), as a distributed machine learning approach, has the potential to address these challenges by sharing only model parameters instead of raw data. Although promising, the dynamics of LEO networks, characterized by the high mobility of satellites and short ground-to-satellite link (GSL) duration, pose unique challenges for FEEL. Notably, frequent model transmission between the satellites and ground incurs prolonged waiting time and large transmission latency. This paper introduces a novel FEEL algorithm, named FEDMEGA, tailored to LEO mega-constellation networks. By integrating inter-satellite links (ISL) for intra-orbit model aggregation, the proposed algorithm significantly reduces the usage of low data rate and intermittent GSL. Our proposed method includes a ring all-reduce based intra-orbit aggregation mechanism, coupled with a network flow-based transmission scheme for global model aggregation, which enhances transmission efficiency. Theoretical convergence analysis is provided to characterize the algorithm performance. Extensive simulations show that our FEDMEGA algorithm outperforms existing satellite FEEL algorithms, exhibiting an approximate 30% improvement in convergence rate. △ Less

Submitted 2 April, 2024; originally announced April 2024.

Comments: 16 pages, 15 figures

Drag Your Noise: Interactive Point-based Editing via Diffusion Semantic Propagation

Authors: Haofeng Liu, Chenshu Xu, Yifei Yang, Lihua Zeng, Shengfeng He

Abstract: Point-based interactive editing serves as an essential tool to complement the controllability of existing generative models. A concurrent work, DragDiffusion, updates the diffusion latent map in response to user inputs, causing global latent map alterations. This results in imprecise preservation of the original content and unsuccessful editing due to gradient vanishing. In contrast, we present Dr… ▽ More Point-based interactive editing serves as an essential tool to complement the controllability of existing generative models. A concurrent work, DragDiffusion, updates the diffusion latent map in response to user inputs, causing global latent map alterations. This results in imprecise preservation of the original content and unsuccessful editing due to gradient vanishing. In contrast, we present DragNoise, offering robust and accelerated editing without retracing the latent map. The core rationale of DragNoise lies in utilizing the predicted noise output of each U-Net as a semantic editor. This approach is grounded in two critical observations: firstly, the bottleneck features of U-Net inherently possess semantically rich features ideal for interactive editing; secondly, high-level semantics, established early in the denoising process, show minimal variation in subsequent stages. Leveraging these insights, DragNoise edits diffusion semantics in a single denoising step and efficiently propagates these changes, ensuring stability and efficiency in diffusion editing. Comparative experiments reveal that DragNoise achieves superior control and semantic retention, reducing the optimization time by over 50% compared to DragDiffusion. Our codes are available at https://github.com/haofengl/DragNoise. △ Less

Submitted 1 April, 2024; originally announced April 2024.

Comments: Accepted by CVPR 2024

Sample Empirical Likelihood Methods for Causal Inference

Authors: Jingyue Huang, Changbao Wu, Leilei Zeng

Abstract: Causal inference is crucial for understanding the true impact of interventions, policies, or actions, enabling informed decision-making and providing insights into the underlying mechanisms that shape our world. In this paper, we establish a framework for the estimation and inference of average treatment effects using a two-sample empirical likelihood function. Two different approaches to incorpor… ▽ More Causal inference is crucial for understanding the true impact of interventions, policies, or actions, enabling informed decision-making and providing insights into the underlying mechanisms that shape our world. In this paper, we establish a framework for the estimation and inference of average treatment effects using a two-sample empirical likelihood function. Two different approaches to incorporating propensity scores are developed. The first approach introduces propensity scores calibrated constraints in addition to the standard model-calibration constraints; the second approach uses the propensity scores to form weighted versions of the model-calibration constraints. The resulting estimators from both approaches are doubly robust. The limiting distributions of the two sample empirical likelihood ratio statistics are derived, facilitating the construction of confidence intervals and hypothesis tests for the average treatment effect. Bootstrap methods for constructing sample empirical likelihood ratio confidence intervals are also discussed for both approaches. Finite sample performances of the methods are investigated through simulation studies. △ Less

Submitted 24 March, 2024; originally announced March 2024.

An improved light-cone harmonic oscillator model for the $φ$-meson longitudinal leading-twist light-cone distribution amplitude

Authors: Dan-Dan Hu, Xing-Gang Wu, Long Zeng, Hai-Bing Fu, Tao Zhong

Abstract: In the present paper, we study the properties of $φ$-meson longitudinal leading-twist light-cone distribution amplitude $φ_{2;φ}^{\|}(x,μ)$ by starting from a light-cone harmonic oscillator model for its wavefunction. To fix the input parameters, we derive the first ten $ξ$-moments of $φ_{2;φ}^{\|}(x,μ)$ by using the QCD sum rules approach under the background field theory. The shape of… ▽ More In the present paper, we study the properties of $φ$-meson longitudinal leading-twist light-cone distribution amplitude $φ_{2;φ}^{\|}(x,μ)$ by starting from a light-cone harmonic oscillator model for its wavefunction. To fix the input parameters, we derive the first ten $ξ$-moments of $φ_{2;φ}^{\|}(x,μ)$ by using the QCD sum rules approach under the background field theory. The shape of $φ_{2;φ}^{\|}(x,μ=2~{\rm GeV})$ tends to be a single-peak behavior, which is consistent with the latest Lattice QCD result. As an application, we derive the $D^+_s \to φ$ transition form factors (TFFs) by using the light-cone sum rules approach. At the large recoil point, we obtain $A_1(0) = 0.512_{-0.020}^{+0.030}$, $A_2(0) = 0.402_{-0.067}^{+0.078}$, $A_0(0) = 0.596_{-0.020}^{+0.025}$ and $V(0) = 0.882_{-0.036}^{+0.040}$. As for the two typical ratios $γ_V$ and $γ_2$, we obtain $γ_V = 1.723_{-0.021}^{+0.023}$ and $γ_2 = 0.785_{-0.104}^{+0.100}$. After extrapolating those TFFs to the physically allowable region, we then obtain the transverse, longitudinal and total decay widths for semi-leptonic decay $D^+_s\toφ\ell^+ν_{\ell}$. Then the branching fractions are ${\cal B}(D^+_s\to φe^+ν_e) = (2.367_{-0.132}^{+0.256})\times 10^{-3}$ and ${\cal B}(D^+_s\to φμ^+ν_μ) = (2.349_{-0.132}^{+0.255})\times 10^{-3}$, which show good agreement with the data issued by the BESIII, the CLEO, and the BABAR Collaborations. We finally calculate $D^+_s\toφ\ell^+ ν_\ell$ polarization and asymmetry parameters. △ Less

Submitted 15 March, 2024; originally announced March 2024.

Comments: 32 pages, 7 figures, comments welcome

SD-Net: Symmetric-Aware Keypoint Prediction and Domain Adaptation for 6D Pose Estimation In Bin-picking Scenarios

Authors: Ding-Tao Huang, En-Te Lin, Lipeng Chen, Li-Fu Liu, Long Zeng

Abstract: Despite the success in 6D pose estimation in bin-picking scenarios, existing methods still struggle to produce accurate prediction results for symmetry objects and real world scenarios. The primary bottlenecks include 1) the ambiguity keypoints caused by object symmetries; 2) the domain gap between real and synthetic data. To circumvent these problem, we propose a new 6D pose estimation network wi… ▽ More Despite the success in 6D pose estimation in bin-picking scenarios, existing methods still struggle to produce accurate prediction results for symmetry objects and real world scenarios. The primary bottlenecks include 1) the ambiguity keypoints caused by object symmetries; 2) the domain gap between real and synthetic data. To circumvent these problem, we propose a new 6D pose estimation network with symmetric-aware keypoint prediction and self-training domain adaptation (SD-Net). SD-Net builds on pointwise keypoint regression and deep hough voting to perform reliable detection keypoint under clutter and occlusion. Specifically, at the keypoint prediction stage, we designe a robust 3D keypoints selection strategy considering the symmetry class of objects and equivalent keypoints, which facilitate locating 3D keypoints even in highly occluded scenes. Additionally, we build an effective filtering algorithm on predicted keypoint to dynamically eliminate multiple ambiguity and outlier keypoint candidates. At the domain adaptation stage, we propose the self-training framework using a student-teacher training scheme. To carefully distinguish reliable predictions, we harnesses a tailored heuristics for 3D geometry pseudo labelling based on semi-chamfer distance. On public Sil'eane dataset, SD-Net achieves state-of-the-art results, obtaining an average precision of 96%. Testing learning and generalization abilities on public Parametric datasets, SD-Net is 8% higher than the state-of-the-art method. The code is available at https://github.com/dingthuang/SD-Net. △ Less

Submitted 14 March, 2024; originally announced March 2024.