-
Graph Representation Learning via Causal Diffusion for Out-of-Distribution Recommendation
Authors:
Chu Zhao,
Enneng Yang,
Yuliang Liang,
Pengxiang Lan,
Yuting Liu,
Jianzhe Zhao,
Guibing Guo,
Xingwei Wang
Abstract:
Graph Neural Networks (GNNs)-based recommendation algorithms typically assume that training and testing data are drawn from independent and identically distributed (IID) spaces. However, this assumption often fails in the presence of out-of-distribution (OOD) data, resulting in significant performance degradation. In this study, we construct a Structural Causal Model (SCM) to analyze interaction d…
▽ More
Graph Neural Networks (GNNs)-based recommendation algorithms typically assume that training and testing data are drawn from independent and identically distributed (IID) spaces. However, this assumption often fails in the presence of out-of-distribution (OOD) data, resulting in significant performance degradation. In this study, we construct a Structural Causal Model (SCM) to analyze interaction data, revealing that environmental confounders (e.g., the COVID-19 pandemic) lead to unstable correlations in GNN-based models, thus impairing their generalization to OOD data. To address this issue, we propose a novel approach, graph representation learning via causal diffusion (CausalDiffRec) for OOD recommendation. This method enhances the model's generalization on OOD data by eliminating environmental confounding factors and learning invariant graph representations. Specifically, we use backdoor adjustment and variational inference to infer the real environmental distribution, thereby eliminating the impact of environmental confounders. This inferred distribution is then used as prior knowledge to guide the representation learning in the reverse phase of the diffusion process to learn the invariant representation. In addition, we provide a theoretical derivation that proves optimizing the objective function of CausalDiffRec can encourage the model to learn environment-invariant graph representations, thereby achieving excellent generalization performance in recommendations under distribution shifts. Our extensive experiments validate the effectiveness of CausalDiffRec in improving the generalization of OOD data, and the average improvement is up to 10.69% on Food, 18.83% on KuaiRec, 22.41% on Yelp2018, and 11.65% on Douban datasets.
△ Less
Submitted 1 August, 2024;
originally announced August 2024.
-
Efficient Prompt Tuning by Multi-Space Projection and Prompt Fusion
Authors:
Pengxiang Lan,
Enneng Yang,
Yuting Liu,
Guibing Guo,
Linying Jiang,
Jianzhe Zhao,
Xingwei Wang
Abstract:
Prompt tuning is a promising method to fine-tune a pre-trained language model without retraining its large-scale parameters. Instead, it attaches a soft prompt to the input text, whereby downstream tasks can be well adapted by merely learning the embeddings of prompt tokens. Nevertheless, existing methods still suffer from two challenges: (i) they are hard to balance accuracy and efficiency. A lon…
▽ More
Prompt tuning is a promising method to fine-tune a pre-trained language model without retraining its large-scale parameters. Instead, it attaches a soft prompt to the input text, whereby downstream tasks can be well adapted by merely learning the embeddings of prompt tokens. Nevertheless, existing methods still suffer from two challenges: (i) they are hard to balance accuracy and efficiency. A longer (shorter) soft prompt generally leads to a better(worse) accuracy but at the cost of more (less) training time. (ii)The performance may not be consistent when adapting to different downstream tasks. We attribute it to the same embedding space but responsible for different requirements of downstream tasks. To address these issues, we propose an Efficient Prompt Tuning method (EPT) by multi-space projection and prompt fusion. Specifically, it decomposes a given soft prompt into a shorter prompt and two low-rank matrices, significantly reducing the training time. Accuracy is also enhanced by leveraging low-rank matrices and the short prompt as additional knowledge sources to enrich the semantics of the original short prompt. In addition, we project the soft prompt into multiple subspaces to improve the performance consistency, and then adaptively learn the combination weights of different spaces through a gating network. Experiments on 13 natural language processing downstream tasks show that our method significantly and consistently outperforms 11 comparison methods with the relative percentage of improvements up to 12.9%, and training time decreased by 14%.
△ Less
Submitted 1 July, 2024; v1 submitted 19 May, 2024;
originally announced May 2024.
-
Elastically-Constrained Meta-Learner for Federated Learning
Authors:
Peng Lan,
Donglai Chen,
Chong Xie,
Keshu Chen,
Jinyuan He,
Juntao Zhang,
Yonghong Chen,
Yan Xu
Abstract:
Federated learning is an approach to collaboratively training machine learning models for multiple parties that prohibit data sharing. One of the challenges in federated learning is non-IID data between clients, as a single model can not fit the data distribution for all clients. Meta-learning, such as Per-FedAvg, is introduced to cope with the challenge. Meta-learning learns shared initial parame…
▽ More
Federated learning is an approach to collaboratively training machine learning models for multiple parties that prohibit data sharing. One of the challenges in federated learning is non-IID data between clients, as a single model can not fit the data distribution for all clients. Meta-learning, such as Per-FedAvg, is introduced to cope with the challenge. Meta-learning learns shared initial parameters for all clients. Each client employs gradient descent to adapt the initialization to local data distributions quickly to realize model personalization. However, due to non-convex loss function and randomness of sampling update, meta-learning approaches have unstable goals in local adaptation for the same client. This fluctuation in different adaptation directions hinders the convergence in meta-learning. To overcome this challenge, we use the historical local adapted model to restrict the direction of the inner loop and propose an elastic-constrained method. As a result, the current round inner loop keeps historical goals and adapts to better solutions. Experiments show our method boosts meta-learning convergence and improves personalization without additional calculation and communication. Our method achieved SOTA on all metrics in three public datasets.
△ Less
Submitted 5 August, 2023; v1 submitted 29 June, 2023;
originally announced June 2023.
-
Walle: An End-to-End, General-Purpose, and Large-Scale Production System for Device-Cloud Collaborative Machine Learning
Authors:
Chengfei Lv,
Chaoyue Niu,
Renjie Gu,
Xiaotang Jiang,
Zhaode Wang,
Bin Liu,
Ziqi Wu,
Qiulin Yao,
Congyu Huang,
Panos Huang,
Tao Huang,
Hui Shu,
Jinde Song,
Bin Zou,
Peng Lan,
Guohuan Xu,
Fei Wu,
Shaojie Tang,
Fan Wu,
Guihai Chen
Abstract:
To break the bottlenecks of mainstream cloud-based machine learning (ML) paradigm, we adopt device-cloud collaborative ML and build the first end-to-end and general-purpose system, called Walle, as the foundation. Walle consists of a deployment platform, distributing ML tasks to billion-scale devices in time; a data pipeline, efficiently preparing task input; and a compute container, providing a c…
▽ More
To break the bottlenecks of mainstream cloud-based machine learning (ML) paradigm, we adopt device-cloud collaborative ML and build the first end-to-end and general-purpose system, called Walle, as the foundation. Walle consists of a deployment platform, distributing ML tasks to billion-scale devices in time; a data pipeline, efficiently preparing task input; and a compute container, providing a cross-platform and high-performance execution environment, while facilitating daily task iteration. Specifically, the compute container is based on Mobile Neural Network (MNN), a tensor compute engine along with the data processing and model execution libraries, which are exposed through a refined Python thread-level virtual machine (VM) to support diverse ML tasks and concurrent task execution. The core of MNN is the novel mechanisms of operator decomposition and semi-auto search, sharply reducing the workload in manually optimizing hundreds of operators for tens of hardware backends and further quickly identifying the best backend with runtime optimization for a computation graph. The data pipeline introduces an on-device stream processing framework to enable processing user behavior data at source. The deployment platform releases ML tasks with an efficient push-then-pull method and supports multi-granularity deployment policies. We evaluate Walle in practical e-commerce application scenarios to demonstrate its effectiveness, efficiency, and scalability. Extensive micro-benchmarks also highlight the superior performance of MNN and the Python thread-level VM. Walle has been in large-scale production use in Alibaba, while MNN has been open source with a broad impact in the community.
△ Less
Submitted 29 May, 2022;
originally announced May 2022.
-
BlazeNeo: Blazing fast polyp segmentation and neoplasm detection
Authors:
Nguyen Sy An,
Phan Ngoc Lan,
Dao Viet Hang,
Dao Van Long,
Tran Quang Trung,
Nguyen Thi Thuy,
Dinh Viet Sang
Abstract:
In recent years, computer-aided automatic polyp segmentation and neoplasm detection have been an emerging topic in medical image analysis, providing valuable support to colonoscopy procedures. Attentions have been paid to improving the accuracy of polyp detection and segmentation. However, not much focus has been given to latency and throughput for performing these tasks on dedicated devices, whic…
▽ More
In recent years, computer-aided automatic polyp segmentation and neoplasm detection have been an emerging topic in medical image analysis, providing valuable support to colonoscopy procedures. Attentions have been paid to improving the accuracy of polyp detection and segmentation. However, not much focus has been given to latency and throughput for performing these tasks on dedicated devices, which can be crucial for practical applications. This paper introduces a novel deep neural network architecture called BlazeNeo, for the task of polyp segmentation and neoplasm detection with an emphasis on compactness and speed while maintaining high accuracy. The model leverages the highly efficient HarDNet backbone alongside lightweight Receptive Field Blocks for computational efficiency, and an auxiliary training mechanism to take full advantage of the training data for the segmentation quality. Our experiments on a challenging dataset show that BlazeNeo achieves improvements in latency and model size while maintaining comparable accuracy against state-of-the-art methods. When deploying on the Jetson AGX Xavier edge device in INT8 precision, our BlazeNeo achieves over 155 fps while yielding the best accuracy among all compared methods.
△ Less
Submitted 28 February, 2022;
originally announced March 2022.
-
NeoUNet: Towards accurate colon polyp segmentation and neoplasm detection
Authors:
Phan Ngoc Lan,
Nguyen Sy An,
Dao Viet Hang,
Dao Van Long,
Tran Quang Trung,
Nguyen Thi Thuy,
Dinh Viet Sang
Abstract:
Automatic polyp segmentation has proven to be immensely helpful for endoscopy procedures, reducing the missing rate of adenoma detection for endoscopists while increasing efficiency. However, classifying a polyp as being neoplasm or not and segmenting it at the pixel level is still a challenging task for doctors to perform in a limited time. In this work, we propose a fine-grained formulation for…
▽ More
Automatic polyp segmentation has proven to be immensely helpful for endoscopy procedures, reducing the missing rate of adenoma detection for endoscopists while increasing efficiency. However, classifying a polyp as being neoplasm or not and segmenting it at the pixel level is still a challenging task for doctors to perform in a limited time. In this work, we propose a fine-grained formulation for the polyp segmentation problem. Our formulation aims to not only segment polyp regions, but also identify those at high risk of malignancy with high accuracy. In addition, we present a UNet-based neural network architecture called NeoUNet, along with a hybrid loss function to solve this problem. Experiments show highly competitive results for NeoUNet on our benchmark dataset compared to existing polyp segmentation models.
△ Less
Submitted 11 July, 2021;
originally announced July 2021.
-
AG-CUResNeSt: A Novel Method for Colon Polyp Segmentation
Authors:
Dinh Viet Sang,
Tran Quang Chung,
Phan Ngoc Lan,
Dao Viet Hang,
Dao Van Long,
Nguyen Thi Thuy
Abstract:
Colorectal cancer is among the most common malignancies and can develop from high-risk colon polyps. Colonoscopy is an effective screening tool to detect and remove polyps, especially in the case of precancerous lesions. However, the missing rate in clinical practice is relatively high due to many factors. The procedure could benefit greatly from using AI models for automatic polyp segmentation, w…
▽ More
Colorectal cancer is among the most common malignancies and can develop from high-risk colon polyps. Colonoscopy is an effective screening tool to detect and remove polyps, especially in the case of precancerous lesions. However, the missing rate in clinical practice is relatively high due to many factors. The procedure could benefit greatly from using AI models for automatic polyp segmentation, which provide valuable insights for improving colon polyp detection. However, precise segmentation is still challenging due to variations of polyps in size, shape, texture, and color. This paper proposes a novel neural network architecture called AG-CUResNeSt, which enhances Coupled UNets using the robust ResNeSt backbone and attention gates. The network is capable of effectively combining multi-level features to yield accurate polyp segmentation. Experimental results on five popular benchmark datasets show that our proposed method achieves state-of-the-art accuracy compared to existing methods.
△ Less
Submitted 1 March, 2022; v1 submitted 2 May, 2021;
originally announced May 2021.
-
DPIVE: A Regionalized Location Obfuscation Scheme with Personalized Privacy Levels
Authors:
Shun Zhang,
Pengfei Lan,
Benfei Duan,
Zhili Chen,
Hong Zhong,
Neal N. Xiong
Abstract:
The popularity of cyber-physical systems is fueling the rapid growth of location-based services. This poses the risk of location privacy disclosure. Effective privacy preservation is foremost for various mobile applications. Recently, geo-indistinguishability and expected inference error are proposed for limiting location leakages. In this paper, we argue that personalization means regionalization…
▽ More
The popularity of cyber-physical systems is fueling the rapid growth of location-based services. This poses the risk of location privacy disclosure. Effective privacy preservation is foremost for various mobile applications. Recently, geo-indistinguishability and expected inference error are proposed for limiting location leakages. In this paper, we argue that personalization means regionalization for geo-indistinguishability, and we propose a regionalized location obfuscation mechanism called DPIVE with personalized utility sensitivities. This substantially corrects the differential and distortion privacy problem of PIVE framework proposed by Yu et al. on NDSS 2017. We develop DPIVE with two phases. In Phase I, we determine disjoint sets by partitioning all possible positions such that different locations in the same set share the Protection Location Set (PLS). In Phase II, we construct a probability distribution matrix in which the rows corresponding to the same PLS have their own sensitivity of utility (PLS diameter). Moreover, by designing QK-means algorithm for more search space in 2-D space, we improve DPIVE with refined location partition and present fine-grained personalization, enabling each location to have its own privacy level endowed with a customized privacy budget. Experiments with two public datasets demonstrate that our mechanisms have the superior performance, typically on skewed locations.
△ Less
Submitted 7 October, 2022; v1 submitted 1 February, 2021;
originally announced February 2021.
-
PMI-based MIMO OFDM PHY Integrated Key Exchange (P-MOPI) Scheme
Authors:
Pang-Chang Lan,
Chih-Yao Wu,
Chia-Han Lee,
Ping-Cheng Yeh,
Chen-Mou Cheng
Abstract:
In the literature, J.-P. Cheng et al. have proposed the MIMO-OFDM PHY integrated (MOPI) scheme for achieving physical-layer security in practice without using any cryptographic ciphers. The MOPI scheme uses channel sounding and physical-layer network coding (PNC) to prevent eavesdroppers from learning the channel state information (CSI). Nevertheless, due to the use of multiple antennas for PNC at…
▽ More
In the literature, J.-P. Cheng et al. have proposed the MIMO-OFDM PHY integrated (MOPI) scheme for achieving physical-layer security in practice without using any cryptographic ciphers. The MOPI scheme uses channel sounding and physical-layer network coding (PNC) to prevent eavesdroppers from learning the channel state information (CSI). Nevertheless, due to the use of multiple antennas for PNC at transmitter and beamforming at receiver, it is not possible to have spatial multiplexing nor use space-time codes in our previous MOPI scheme. In this paper, we propose a variant of the MOPI scheme, called P-MOPI, that works with a cryptographic cipher and utilizes precoding matrix index (PMI) as an efficient key-exchange mechanism. With channel sounding, the PMI is only known between the transmitter and the legal receiver. The shared key can then be used, e.g., as the seed to generate pseudo random bit sequences for securing subsequent transmissions using a stream cipher. By applying the same techniques at independent subcarriers of the OFDM system, the P-MOPI scheme easily allows two communicating parties to exchange over 100 secret bits. As a result, not only secure communication but also the MIMO gain can be guaranteed by using the P-MOPI scheme.
△ Less
Submitted 21 January, 2011;
originally announced January 2011.
-
Simulation of valveless micropump and mode analysis
Authors:
W. P. Lan,
J. S. Chang,
K. C. Wu,
Y. C. Shih
Abstract:
In this work, a 3-D simulation is performed to study for the solid-fluid coupling effect driven by piezoelectric materials and utilizes asymmetric obstacles to control the flow direction. The result of simulation is also verified. For a micropump, it is crucial to find the optimal working frequency which produce maximum net flow rate. The PZT plate vibrates under the first mode, which is symmetr…
▽ More
In this work, a 3-D simulation is performed to study for the solid-fluid coupling effect driven by piezoelectric materials and utilizes asymmetric obstacles to control the flow direction. The result of simulation is also verified. For a micropump, it is crucial to find the optimal working frequency which produce maximum net flow rate. The PZT plate vibrates under the first mode, which is symmetric. Adjusting the working frequency, the maximum flow rate can be obtained. For the micrpump we studied, the optimal working frequency is 3.2K Hz. At higher working frequency, say 20K Hz, the fluid-solid membrane may come out a intermediate mode, which is different from the first mode and the second mode. It is observed that the center of the mode drifts. Meanwhile, the result shows that a phase shift lagging when the excitation force exists in the vibration response. Finally, at even higher working frequency, say 30K Hz, a second vibration mode is observed.
△ Less
Submitted 21 February, 2008;
originally announced February 2008.
