Showing 1–21 of 21 results for author: Ning, L

UserSumBench: A Benchmark Framework for Evaluating User Summarization Approaches

Authors: Chao Wang, Neo Wu, Lin Ning, Luyang Liu, Jun Xie, Shawn O'Banion, Bradley Green

Abstract: Large language models (LLMs) have shown remarkable capabilities in generating user summaries from a long list of raw user activity data. These summaries capture essential user information such as preferences and interests, and therefore are invaluable for LLM-based personalization applications, such as explainable recommender systems. However, the development of new summarization techniques is hin… ▽ More Large language models (LLMs) have shown remarkable capabilities in generating user summaries from a long list of raw user activity data. These summaries capture essential user information such as preferences and interests, and therefore are invaluable for LLM-based personalization applications, such as explainable recommender systems. However, the development of new summarization techniques is hindered by the lack of ground-truth labels, the inherent subjectivity of user summaries, and human evaluation which is often costly and time-consuming. To address these challenges, we introduce \UserSumBench, a benchmark framework designed to facilitate iterative development of LLM-based summarization approaches. This framework offers two key components: (1) A reference-free summary quality metric. We show that this metric is effective and aligned with human preferences across three diverse datasets (MovieLens, Yelp and Amazon Review). (2) A novel robust summarization method that leverages time-hierarchical summarizer and self-critique verifier to produce high-quality summaries while eliminating hallucination. This method serves as a strong baseline for further innovation in summarization techniques. △ Less

Submitted 29 August, 2024; originally announced August 2024.

Joint Universal Adversarial Perturbations with Interpretations

Authors: Liang-bo Ning, Zeyu Dai, Wenqi Fan, Jingran Su, Chao Pan, Luning Wang, Qing Li

Abstract: Deep neural networks (DNNs) have significantly boosted the performance of many challenging tasks. Despite the great development, DNNs have also exposed their vulnerability. Recent studies have shown that adversaries can manipulate the predictions of DNNs by adding a universal adversarial perturbation (UAP) to benign samples. On the other hand, increasing efforts have been made to help users unders… ▽ More Deep neural networks (DNNs) have significantly boosted the performance of many challenging tasks. Despite the great development, DNNs have also exposed their vulnerability. Recent studies have shown that adversaries can manipulate the predictions of DNNs by adding a universal adversarial perturbation (UAP) to benign samples. On the other hand, increasing efforts have been made to help users understand and explain the inner working of DNNs by highlighting the most informative parts (i.e., attribution maps) of samples with respect to their predictions. Moreover, we first empirically find that such attribution maps between benign and adversarial examples have a significant discrepancy, which has the potential to detect universal adversarial perturbations for defending against adversarial attacks. This finding motivates us to further investigate a new research problem: whether there exist universal adversarial perturbations that are able to jointly attack DNNs classifier and its interpretation with malicious desires. It is challenging to give an explicit answer since these two objectives are seemingly conflicting. In this paper, we propose a novel attacking framework to generate joint universal adversarial perturbations (JUAP), which can fool the DNNs model and misguide the inspection from interpreters simultaneously. Comprehensive experiments on various datasets demonstrate the effectiveness of the proposed method JUAP for joint attacks. To the best of our knowledge, this is the first effort to study UAP for jointly attacking both DNNs and interpretations. △ Less

Submitted 3 August, 2024; originally announced August 2024.

MoDE: Effective Multi-task Parameter Efficient Fine-Tuning with a Mixture of Dyadic Experts

Authors: Lin Ning, Harsh Lara, Meiqi Guo, Abhinav Rastogi

Abstract: Parameter-efficient fine-tuning techniques like Low-Rank Adaptation (LoRA) have revolutionized the adaptation of large language models (LLMs) to diverse tasks. Recent efforts have explored mixtures of LoRA modules for multi-task settings. However, our analysis reveals redundancy in the down-projection matrices of these architectures. This observation motivates our proposed method, Mixture of Dyadi… ▽ More Parameter-efficient fine-tuning techniques like Low-Rank Adaptation (LoRA) have revolutionized the adaptation of large language models (LLMs) to diverse tasks. Recent efforts have explored mixtures of LoRA modules for multi-task settings. However, our analysis reveals redundancy in the down-projection matrices of these architectures. This observation motivates our proposed method, Mixture of Dyadic Experts (MoDE), which introduces a novel design for efficient multi-task adaptation. This is done by sharing the down-projection matrix across tasks and employing atomic rank-one adapters, coupled with routers that allow more sophisticated task-level specialization. Our design allows for more fine-grained mixing, thereby increasing the model's ability to jointly handle multiple tasks. We evaluate MoDE on the Supernatural Instructions (SNI) benchmark consisting of a diverse set of 700+ tasks and demonstrate that it outperforms state-of-the-art multi-task parameter-efficient fine-tuning (PEFT) methods, without introducing additional parameters. Our findings contribute to a deeper understanding of parameter efficiency in multi-task LLM adaptation and provide a practical solution for deploying high-performing, lightweight models. △ Less

Submitted 2 August, 2024; originally announced August 2024.

A Survey of Mamba

Authors: Haohao Qu, Liangbo Ning, Rui An, Wenqi Fan, Tyler Derr, Hui Liu, Xin Xu, Qing Li

Abstract: As one of the most representative DL techniques, Transformer architecture has empowered numerous advanced models, especially the large language models (LLMs) that comprise billions of parameters, becoming a cornerstone in deep learning. Despite the impressive achievements, Transformers still face inherent limitations, particularly the time-consuming inference resulting from the quadratic computati… ▽ More As one of the most representative DL techniques, Transformer architecture has empowered numerous advanced models, especially the large language models (LLMs) that comprise billions of parameters, becoming a cornerstone in deep learning. Despite the impressive achievements, Transformers still face inherent limitations, particularly the time-consuming inference resulting from the quadratic computation complexity of attention calculation. Recently, a novel architecture named Mamba, drawing inspiration from classical state space models (SSMs), has emerged as a promising alternative for building foundation models, delivering comparable modeling abilities to Transformers while preserving near-linear scalability concerning sequence length. This has sparked an increasing number of studies actively exploring Mamba's potential to achieve impressive performance across diverse domains. Given such rapid evolution, there is a critical need for a systematic review that consolidates existing Mamba-empowered models, offering a comprehensive understanding of this emerging model architecture. In this survey, we therefore conduct an in-depth investigation of recent Mamba-associated studies, covering three main aspects: the advancements of Mamba-based models, the techniques of adapting Mamba to diverse data, and the applications where Mamba can excel. Specifically, we first review the foundational knowledge of various representative deep learning models and the details of Mamba-1&2 as preliminaries. Then, to showcase the significance of Mamba for AI, we comprehensively review the related studies focusing on Mamba models' architecture design, data adaptability, and applications. Finally, we present a discussion of current limitations and explore various promising research directions to provide deeper insights for future investigations. △ Less

Submitted 22 August, 2024; v1 submitted 2 August, 2024; originally announced August 2024.

A study of why we need to reassess full reference image quality assessment with medical images

Authors: Anna Breger, Ander Biguri, Malena Sabaté Landman, Ian Selby, Nicole Amberg, Elisabeth Brunner, Janek Gröhl, Sepideh Hatamikia, Clemens Karner, Lipeng Ning, Sören Dittmer, Michael Roberts, AIX-COVNET Collaboration, Carola-Bibiane Schönlieb

Abstract: Image quality assessment (IQA) is not just indispensable in clinical practice to ensure high standards, but also in the development stage of novel algorithms that operate on medical images with reference data. This paper provides a structured and comprehensive collection of examples where the two most common full reference (FR) image quality measures prove to be unsuitable for the assessment of no… ▽ More Image quality assessment (IQA) is not just indispensable in clinical practice to ensure high standards, but also in the development stage of novel algorithms that operate on medical images with reference data. This paper provides a structured and comprehensive collection of examples where the two most common full reference (FR) image quality measures prove to be unsuitable for the assessment of novel algorithms using different kinds of medical images, including real-world MRI, CT, OCT, X-Ray, digital pathology and photoacoustic imaging data. In particular, the FR-IQA measures PSNR and SSIM are known and tested for working successfully in many natural imaging tasks, but discrepancies in medical scenarios have been noted in the literature. Inconsistencies arising in medical images are not surprising, as they have very different properties than natural images which have not been targeted nor tested in the development of the mentioned measures, and therefore might imply wrong judgement of novel methods for medical images. Therefore, improvement is urgently needed in particular in this era of AI to increase explainability, reproducibility and generalizability in machine learning for medical imaging and beyond. On top of the pitfalls we will provide ideas for future research as well as suggesting guidelines for the usage of FR-IQA measures applied to medical images. △ Less

Submitted 29 May, 2024; originally announced May 2024.

A Survey on RAG Meeting LLMs: Towards Retrieval-Augmented Large Language Models

Authors: Wenqi Fan, Yujuan Ding, Liangbo Ning, Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, Qing Li

Abstract: As one of the most advanced techniques in AI, Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge, providing huge convenience for numerous tasks. Particularly in the era of AI-Generated Content (AIGC), the powerful capacity of retrieval in providing additional knowledge enables RAG to assist existing generative AI in producing high-quality outputs. Recently, L… ▽ More As one of the most advanced techniques in AI, Retrieval-Augmented Generation (RAG) can offer reliable and up-to-date external knowledge, providing huge convenience for numerous tasks. Particularly in the era of AI-Generated Content (AIGC), the powerful capacity of retrieval in providing additional knowledge enables RAG to assist existing generative AI in producing high-quality outputs. Recently, Large Language Models (LLMs) have demonstrated revolutionary abilities in language understanding and generation, while still facing inherent limitations, such as hallucinations and out-of-date internal knowledge. Given the powerful abilities of RAG in providing the latest and helpful auxiliary information, Retrieval-Augmented Large Language Models (RA-LLMs) have emerged to harness external and authoritative knowledge bases, rather than solely relying on the model's internal knowledge, to augment the generation quality of LLMs. In this survey, we comprehensively review existing research studies in RA-LLMs, covering three primary technical perspectives: architectures, training strategies, and applications. As the preliminary knowledge, we briefly introduce the foundations and recent advances of LLMs. Then, to illustrate the practical significance of RAG for LLMs, we systematically review mainstream relevant work by their architectures, training strategies, and application areas, detailing specifically the challenges of each and the corresponding capabilities of RA-LLMs. Finally, to deliver deeper insights, we discuss current limitations and several promising directions for future research. Updated information about this survey can be found at https://advanced-recommender-systems.github.io/RAG-Meets-LLMs/ △ Less

Submitted 17 June, 2024; v1 submitted 9 May, 2024; originally announced May 2024.

Comments: This is the long version of the corresponding survey paper accepted by KDD2024

User-LLM: Efficient LLM Contextualization with User Embeddings

Authors: Lin Ning, Luyang Liu, Jiaxing Wu, Neo Wu, Devora Berlowitz, Sushant Prakash, Bradley Green, Shawn O'Banion, Jun Xie

Abstract: Large language models (LLMs) have revolutionized natural language processing. However, effectively incorporating complex and potentially noisy user interaction data remains a challenge. To address this, we propose User-LLM, a novel framework that leverages user embeddings to contextualize LLMs. These embeddings, distilled from diverse user interactions using self-supervised pretraining, capture la… ▽ More Large language models (LLMs) have revolutionized natural language processing. However, effectively incorporating complex and potentially noisy user interaction data remains a challenge. To address this, we propose User-LLM, a novel framework that leverages user embeddings to contextualize LLMs. These embeddings, distilled from diverse user interactions using self-supervised pretraining, capture latent user preferences and their evolution over time. We integrate these user embeddings with LLMs through cross-attention and soft-prompting, enabling LLMs to dynamically adapt to user context. Our comprehensive experiments on MovieLens, Amazon Review, and Google Local Review datasets demonstrate significant performance gains across various tasks. Notably, our approach outperforms text-prompt-based contextualization on long sequence tasks and tasks that require deep user understanding while being computationally efficient. We further incorporate Perceiver layers to streamline the integration between user encoders and LLMs, reducing computational demands. △ Less

Submitted 21 February, 2024; originally announced February 2024.

SlicerTMS: Real-Time Visualization of Transcranial Magnetic Stimulation for Mental Health Treatment

Authors: Loraine Franke, Tae Young Park, Jie Luo, Yogesh Rathi, Steve Pieper, Lipeng Ning, Daniel Haehn

Abstract: We present a real-time visualization system for Transcranial Magnetic Stimulation (TMS), a non-invasive neuromodulation technique for treating various brain disorders and mental health diseases. Our solution targets the current challenges of slow and labor-intensive practices in treatment planning. Integrating Deep Learning (DL), our system rapidly predicts electric field (E-field) distributions i… ▽ More We present a real-time visualization system for Transcranial Magnetic Stimulation (TMS), a non-invasive neuromodulation technique for treating various brain disorders and mental health diseases. Our solution targets the current challenges of slow and labor-intensive practices in treatment planning. Integrating Deep Learning (DL), our system rapidly predicts electric field (E-field) distributions in 0.2 seconds for precise and effective brain stimulation. The core advancement lies in our tool's real-time neuronavigation visualization capabilities, which support clinicians in making more informed decisions quickly and effectively. We assess our system's performance through three studies: First, a real-world use case scenario in a clinical setting, providing concrete feedback on applicability and usability in a practical environment. Second, a comparative analysis with another TMS tool focusing on computational efficiency across various hardware platforms. Lastly, we conducted an expert user study to measure usability and influence in optimizing TMS treatment planning. The system is openly available for community use and further development on GitHub: \url{https://github.com/lorifranke/SlicerTMS}. △ Less

Submitted 12 March, 2024; v1 submitted 10 May, 2023; originally announced May 2023.

Comments: 11 pages, 4 figures, 2 tables, MICCAI

Molecular design method based on novel molecular representation and variational auto-encoder

Authors: Li Kai, Li Ning, Zhang Wei, Gao Ming

Abstract: Based on the traditional VAE, a novel neural network model is presented, with the latest molecular representation, SELFIES, to improve the effect of generating new molecules. In this model, multi-layer convolutional network and Fisher information are added to the original encoding layer to learn the data characteristics and guide the encoding process, which makes the features of the data hiding la… ▽ More Based on the traditional VAE, a novel neural network model is presented, with the latest molecular representation, SELFIES, to improve the effect of generating new molecules. In this model, multi-layer convolutional network and Fisher information are added to the original encoding layer to learn the data characteristics and guide the encoding process, which makes the features of the data hiding layer more aggregated, and integrates the Long Short Term Memory neural network (LSTM) into the decoding layer for better data generation, which effectively solves the degradation phenomenon generated by the encoding layer and decoding layer of the original VAE model. Through experiments on zinc molecular data sets, it is found that the similarity in the new VAE is 8.47% higher than that of the original ones. SELFIES are better at generating a variety of molecules than the traditional molecular representation, SELFIES. Experiments have shown that using SELFIES and the new VAE model presented in this paper can improve the effectiveness of generating new molecules. △ Less

Submitted 20 February, 2023; originally announced May 2023.

Comments: 13 pages, 7 figures, conference: NIAI

Journal ref: 4th International Conference on Natural Language Processing, Information Retrieval and AI (NIAI 2023), Volume 13, Number 03, February 2023, pp. 23-35, 2023. CS & IT - CSCP 2023

Minimum-entropy causal inference and its application in brain network analysis

Authors: Lipeng Ning

Abstract: Identification of the causal relationship between multivariate time series is a ubiquitous problem in data science. Granger causality measure (GCM) and conditional Granger causality measure (cGCM) are widely used statistical methods for causal inference and effective connectivity analysis in neuroimaging research. Both GCM and cGCM have frequency-domain formulations that are developed based on a h… ▽ More Identification of the causal relationship between multivariate time series is a ubiquitous problem in data science. Granger causality measure (GCM) and conditional Granger causality measure (cGCM) are widely used statistical methods for causal inference and effective connectivity analysis in neuroimaging research. Both GCM and cGCM have frequency-domain formulations that are developed based on a heuristic algorithm for matrix decompositions. The goal of this work is to generalize GCM and cGCM measures and their frequency-domain formulations by using a theoretic framework for minimum entropy (ME) estimation. The proposed ME-estimation method extends the classical theory of minimum mean squared error (MMSE) estimation for stochastic processes. It provides three formulations of cGCM that include Geweke's original time-domain cGCM as a special case. But all three frequency-domain formulations of cGCM are different from previous methods. Experimental results based on simulations have shown that one of the proposed frequency-domain cGCM has enhanced sensitivity and specificity in detecting network connections compared to other methods. In an example based on in vivo functional magnetic resonance imaging, the proposed frequency-domain measure cGCM can significantly enhance the consistency between the structural and effective connectivity of human brain networks. △ Less

Submitted 7 September, 2022; originally announced September 2022.

Mixed Federated Learning: Joint Decentralized and Centralized Learning

Authors: Sean Augenstein, Andrew Hard, Lin Ning, Karan Singhal, Satyen Kale, Kurt Partridge, Rajiv Mathews

Abstract: Federated learning (FL) enables learning from decentralized privacy-sensitive data, with computations on raw data confined to take place at edge clients. This paper introduces mixed FL, which incorporates an additional loss term calculated at the coordinating server (while maintaining FL's private data restrictions). There are numerous benefits. For example, additional datacenter data can be lever… ▽ More Federated learning (FL) enables learning from decentralized privacy-sensitive data, with computations on raw data confined to take place at edge clients. This paper introduces mixed FL, which incorporates an additional loss term calculated at the coordinating server (while maintaining FL's private data restrictions). There are numerous benefits. For example, additional datacenter data can be leveraged to jointly learn from centralized (datacenter) and decentralized (federated) training data and better match an expected inference data distribution. Mixed FL also enables offloading some intensive computations (e.g., embedding regularization) to the server, greatly reducing communication and client computation load. For these and other mixed FL use cases, we present three algorithms: PARALLEL TRAINING, 1-WAY GRADIENT TRANSFER, and 2-WAY GRADIENT TRANSFER. We state convergence bounds for each, and give intuition on which are suited to particular mixed FL problems. Finally we perform extensive experiments on three tasks, demonstrating that mixed FL can blend training data to achieve an oracle's accuracy on an inference distribution, and can reduce communication and computation overhead by over 90%. Our experiments confirm theoretical predictions of how algorithms perform under different mixed FL problem settings. △ Less

Submitted 24 June, 2022; v1 submitted 26 May, 2022; originally announced May 2022.

Comments: 36 pages, 12 figures. Image resolutions reduced for easier downloading

What Do We Mean by Generalization in Federated Learning?

Authors: Honglin Yuan, Warren Morningstar, Lin Ning, Karan Singhal

Abstract: Federated learning data is drawn from a distribution of distributions: clients are drawn from a meta-distribution, and their data are drawn from local data distributions. Thus generalization studies in federated learning should separate performance gaps from unseen client data (out-of-sample gap) from performance gaps from unseen client distributions (participation gap). In this work, we propose a… ▽ More Federated learning data is drawn from a distribution of distributions: clients are drawn from a meta-distribution, and their data are drawn from local data distributions. Thus generalization studies in federated learning should separate performance gaps from unseen client data (out-of-sample gap) from performance gaps from unseen client distributions (participation gap). In this work, we propose a framework for disentangling these performance gaps. Using this framework, we observe and explain differences in behavior across natural and synthetic federated datasets, indicating that dataset synthesis strategy can be important for realistic simulations of generalization in federated learning. We propose a semantic synthesis strategy that enables realistic simulation without naturally-partitioned data. Informed by our findings, we call out community suggestions for future federated learning works. △ Less

Submitted 16 March, 2022; v1 submitted 27 October, 2021; originally announced October 2021.

Comments: Accepted to ICLR 2022. Code repository see https://bit.ly/fl-generalization

Learning Federated Representations and Recommendations with Limited Negatives

Authors: Lin Ning, Karan Singhal, Ellie X. Zhou, Sushant Prakash

Abstract: Deep retrieval models are widely used for learning entity representations and recommendations. Federated learning provides a privacy-preserving way to train these models without requiring centralization of user data. However, federated deep retrieval models usually perform much worse than their centralized counterparts due to non-IID (independent and identically distributed) training data on clien… ▽ More Deep retrieval models are widely used for learning entity representations and recommendations. Federated learning provides a privacy-preserving way to train these models without requiring centralization of user data. However, federated deep retrieval models usually perform much worse than their centralized counterparts due to non-IID (independent and identically distributed) training data on clients, an intrinsic property of federated learning that limits negatives available for training. We demonstrate that this issue is distinct from the commonly studied client drift problem. This work proposes batch-insensitive losses as a way to alleviate the non-IID negatives issue for federated movie recommendations. We explore a variety of techniques and identify that batch-insensitive losses can effectively improve the performance of federated deep retrieval models, increasing the relative recall of the federated model by up to 93.15% and reducing the relative gap in recall between it and a centralized model from 27.22% - 43.14% to 0.53% - 2.42%. We also open-source our code framework to accelerate further research and applications of federated deep retrieval models. △ Less

Submitted 2 November, 2021; v1 submitted 17 August, 2021; originally announced August 2021.

LAC-Nav: Collision-Free Mutiagent Navigation Based on The Local Action Cells

Authors: Li Ning, Yong Zhang

Abstract: Collision avoidance is one of the most primary requirement in the decentralized multiagent navigations: while the agents are moving towards their own targets, attentions should be paid to avoid the collisions with the others. In this paper, we introduce the concept of local action cell, which provides for each agent a set of velocities that are safe to perform. Based on the realtime updated local… ▽ More Collision avoidance is one of the most primary requirement in the decentralized multiagent navigations: while the agents are moving towards their own targets, attentions should be paid to avoid the collisions with the others. In this paper, we introduce the concept of local action cell, which provides for each agent a set of velocities that are safe to perform. Based on the realtime updated local action cells, we propose the LAC-Nav approach to navigate the agent with the properly selected velocity; and furthermore, we coupled the local action cell with an adaptive learning framework, in which the effect of selections are evaluated and used as the references for making decisions in the following updates. Through the experiments for three commonly considered scenarios, we demonstrated the efficiency of the proposed approaches, with the comparison to several widely studied strategies. △ Less

Submitted 11 November, 2019; originally announced November 2019.

In-Place Zero-Space Memory Protection for CNN

Authors: Hui Guan, Lin Ning, Zhen Lin, Xipeng Shen, Huiyang Zhou, Seung-Hwan Lim

Abstract: Convolutional Neural Networks (CNN) are being actively explored for safety-critical applications such as autonomous vehicles and aerospace, where it is essential to ensure the reliability of inference results in the presence of possible memory faults. Traditional methods such as error correction codes (ECC) and Triple Modular Redundancy (TMR) are CNN-oblivious and incur substantial memory overhead… ▽ More Convolutional Neural Networks (CNN) are being actively explored for safety-critical applications such as autonomous vehicles and aerospace, where it is essential to ensure the reliability of inference results in the presence of possible memory faults. Traditional methods such as error correction codes (ECC) and Triple Modular Redundancy (TMR) are CNN-oblivious and incur substantial memory overhead and energy cost. This paper introduces in-place zero-space ECC assisted with a new training scheme weight distribution-oriented training. The new method provides the first known zero space cost memory protection for CNNs without compromising the reliability offered by traditional ECC. △ Less

Submitted 31 October, 2019; originally announced October 2019.

Comments: Accepted in NeurIPS'19

PyramNet: Point Cloud Pyramid Attention Network and Graph Embedding Module for Classification and Segmentation

Authors: Kang Zhiheng, Li Ning

Abstract: With the tide of artificial intelligence, we try to apply deep learning to understand 3D data. Point cloud is an important 3D data structure, which can accurately and directly reflect the real world. In this paper, we propose a simple and effective network, which is named PyramNet, suites for point cloud object classification and semantic segmentation in 3D scene. We design two new operators: Grap… ▽ More With the tide of artificial intelligence, we try to apply deep learning to understand 3D data. Point cloud is an important 3D data structure, which can accurately and directly reflect the real world. In this paper, we propose a simple and effective network, which is named PyramNet, suites for point cloud object classification and semantic segmentation in 3D scene. We design two new operators: Graph Embedding Module(GEM) and Pyramid Attention Network(PAN). Specifically, GEM projects point cloud onto the graph and practices the covariance matrix to explore the relationship between points, so as to improve the local feature expression ability of the model. PAN assigns some strong semantic features to each point to retain fine geometric features as much as possible. Furthermore, we provide extensive evaluation and analysis for the effectiveness of PyramNet. Empirically, we evaluate our model on ModelNet40, ShapeNet and S3DIS. △ Less

Submitted 30 September, 2019; v1 submitted 7 June, 2019; originally announced June 2019.

Comments: Accepted for presentation at ICONIP2019

Regularization of covariance matrices on Riemannian manifolds using linear systems

Authors: Lipeng Ning

Abstract: We propose an approach to use the state covariance of linear systems to track time-varying covariance matrices of non-stationary time series. Following concepts from Riemmanian geometry, we investigate three types of covariance paths obtained by using different quadratic regularizations of system matrices. The first quadratic form induces the geodesics based on the Bures-Wasserstein metric from op… ▽ More We propose an approach to use the state covariance of linear systems to track time-varying covariance matrices of non-stationary time series. Following concepts from Riemmanian geometry, we investigate three types of covariance paths obtained by using different quadratic regularizations of system matrices. The first quadratic form induces the geodesics based on the Bures-Wasserstein metric from optimal mass transport theory and quantum mechanics. The second type of quadratic form leads to the geodesics based on the Fisher-Rao metric from information geometry. In the process, we introduce a fluid-mechanics interpretation of the Fisher-Rao metric for multivariate Gaussian distributions. A main contribution of this work is the introduction of the third type of covariance paths which are steered by linear system matrices with rotating eigenspace. We provide theoretical results on the existence and uniqueness of this type of covariance paths. The three types of covariance paths are compared using two examples with synthetic data and real data based on functional magnetic resonance imaging, respectively. △ Less

Submitted 29 May, 2018; originally announced May 2018.

Comments: 30 pages, 4 figures

Influence Maximization under The Non-progressive Linear Threshold Model

Authors: T. -H. Hubert Chan, Li Ning

Abstract: In the problem of influence maximization in information networks, the objective is to choose a set of initially active nodes subject to some budget constraints such that the expected number of active nodes over time is maximized. The linear threshold model has been introduced to study the opinion cascading behavior, for instance, the spread of products and innovations. In this paper, we we extends… ▽ More In the problem of influence maximization in information networks, the objective is to choose a set of initially active nodes subject to some budget constraints such that the expected number of active nodes over time is maximized. The linear threshold model has been introduced to study the opinion cascading behavior, for instance, the spread of products and innovations. In this paper, we we extends the classic linear threshold model [18] to capture the non-progressive be- havior. The information maximization problem under our model is proved to be NP-Hard, even for the case when the underlying network has no directed cycles. The first result of this paper is negative. In general, the objective function of the extended linear threshold model is no longer submodular, and hence the hill climbing approach that is commonly used in the existing studies is not applicable. Next, as the main result of this paper, we prove that if the underlying information network is directed acyclic, the objective function is submodular (and monotone). Therefore, in directed acyclic networks with a specified budget we can achieve 1/2 -approximation on maximizing the number of active nodes over a certain period of time by a deterministic algorithm, and achieve the (1 - 1/e )-approximation by a randomized algorithm. △ Less

Submitted 1 April, 2015; originally announced April 2015.

Uniform Information Exchange in Multi-channel Wireless Ad Hoc Networks

Authors: Li Ning, Dongxiao Yu, Yong Zhang, Yuexuan Wang, Francis C. M. Lau, Shenzhong Feng

Abstract: In the information exchange problem, k packets that are initially maintained by k nodes need to be disseminated to the whole network as quickly as possible. We consider this problem in single-hop multi- channel networks of n nodes, and propose a uniform protocol that with high probability accomplishes the dissemination in O(k/F + F \cdot log n) rounds, assuming F available channels and collision d… ▽ More In the information exchange problem, k packets that are initially maintained by k nodes need to be disseminated to the whole network as quickly as possible. We consider this problem in single-hop multi- channel networks of n nodes, and propose a uniform protocol that with high probability accomplishes the dissemination in O(k/F + F \cdot log n) rounds, assuming F available channels and collision detection. This result is asymptotically optimal when k is large (k \geq F^2 \cdot log n). To our knowledge, this is the first uniform protocol for information exchange in multi-channel networks. △ Less

Submitted 30 March, 2015; originally announced March 2015.

Incubators vs Zombies: Fault-Tolerant, Short, Thin and Lanky Spanners for Doubling Metrics

Authors: T-H. Hubert Chan, Mingfei Li, Li Ning

Abstract: Recently Elkin and Solomon gave a construction of spanners for doubling metrics that has constant maximum degree, hop-diameter O(log n) and lightness O(log n) (i.e., weight O(log n)w(MST). This resolves a long standing conjecture proposed by Arya et al. in a seminal STOC 1995 paper. However, Elkin and Solomon's spanner construction is extremely complicated; we offer a simple alternative construc… ▽ More Recently Elkin and Solomon gave a construction of spanners for doubling metrics that has constant maximum degree, hop-diameter O(log n) and lightness O(log n) (i.e., weight O(log n)w(MST). This resolves a long standing conjecture proposed by Arya et al. in a seminal STOC 1995 paper. However, Elkin and Solomon's spanner construction is extremely complicated; we offer a simple alternative construction that is very intuitive and is based on the standard technique of net tree with cross edges. Indeed, our approach can be readily applied to our previous construction of k-fault tolerant spanners (ICALP 2012) to achieve k-fault tolerance, maximum degree O(k^2), hop-diameter O(log n) and lightness O(k^3 log n). △ Less

Submitted 3 July, 2012; originally announced July 2012.

ACM Class: F.2.2

(1+epsilon)-Distance Oracle for Planar Labeled Graph

Authors: Mingfei Li, Christoffer Ma, Li Ning

Abstract: Given a vertex-labeled graph, each vertex $v$ is attached with a label from a set of labels. The vertex-label query desires the length of the shortest path from the given vertex to the set of vertices with the given label. We show how to construct an oracle if the given graph is planar, such that $O(\frac{1}εn\log n)$ storing space is needed, and any vertex-label query could be answered in… ▽ More Given a vertex-labeled graph, each vertex $v$ is attached with a label from a set of labels. The vertex-label query desires the length of the shortest path from the given vertex to the set of vertices with the given label. We show how to construct an oracle if the given graph is planar, such that $O(\frac{1}εn\log n)$ storing space is needed, and any vertex-label query could be answered in $O(\frac{1}ε\log n\log ρ)$ time with stretch $1+ε$. $ρ$ is the radius of the given graph, which is half of the diameter. For the case that $ρ= O(\log n)$, we construct an oracle that achieves $O(\log n)$ query time, without changing the order of storing space. △ Less

Submitted 31 January, 2012; v1 submitted 29 December, 2011; originally announced December 2011.