Showing 1–12 of 12 results for author: Theodoropoulos, C

Evaluating the Predictive Features of Person-Centric Knowledge Graph Embeddings: Unfolding Ablation Studies

Authors: Christos Theodoropoulos, Natasha Mulligan, Joao Bettencourt-Silva

Abstract: Developing novel predictive models with complex biomedical information is challenging due to various idiosyncrasies related to heterogeneity, standardization or sparseness of the data. We previously introduced a person-centric ontology to organize information about individual patients, and a representation learning framework to extract person-centric knowledge graphs (PKGs) and to train Graph Neur… ▽ More Developing novel predictive models with complex biomedical information is challenging due to various idiosyncrasies related to heterogeneity, standardization or sparseness of the data. We previously introduced a person-centric ontology to organize information about individual patients, and a representation learning framework to extract person-centric knowledge graphs (PKGs) and to train Graph Neural Networks (GNNs). In this paper, we propose a systematic approach to examine the results of GNN models trained with both structured and unstructured information from the MIMIC-III dataset. Through ablation studies on different clinical, demographic, and social data, we show the robustness of this approach in identifying predictive features in PKGs for the task of readmission prediction. △ Less

Submitted 29 August, 2024; v1 submitted 27 August, 2024; originally announced August 2024.

Comments: Published in the 34th Medical Informatics Europe Conference

Journal ref: Studies in health technology and informatics vol. 316 (2024): 575-579

Fast-and-Frugal Text-Graph Transformers are Effective Link Predictors

Authors: Andrei C. Coman, Christos Theodoropoulos, Marie-Francine Moens, James Henderson

Abstract: Link prediction models can benefit from incorporating textual descriptions of entities and relations, enabling fully inductive learning and flexibility in dynamic graphs. We address the challenge of also capturing rich structured information about the local neighbourhood of entities and their relations, by introducing a Transformer-based approach that effectively integrates textual descriptions wi… ▽ More Link prediction models can benefit from incorporating textual descriptions of entities and relations, enabling fully inductive learning and flexibility in dynamic graphs. We address the challenge of also capturing rich structured information about the local neighbourhood of entities and their relations, by introducing a Transformer-based approach that effectively integrates textual descriptions with graph structure, reducing the reliance on resource-intensive text encoders. Our experiments on three challenging datasets show that our Fast-and-Frugal Text-Graph (FnF-TG) Transformers achieve superior performance compared to the previous state-of-the-art methods, while maintaining efficiency and scalability. △ Less

Submitted 13 August, 2024; originally announced August 2024.

Enhancing Biomedical Knowledge Discovery for Diseases: An Open-Source Framework Applied on Rett Syndrome and Alzheimer's Disease

Authors: Christos Theodoropoulos, Andrei Catalin Coman, James Henderson, Marie-Francine Moens

Abstract: The ever-growing volume of biomedical publications creates a critical need for efficient knowledge discovery. In this context, we introduce an open-source end-to-end framework designed to construct knowledge around specific diseases directly from raw text. To facilitate research in disease-related knowledge discovery, we create two annotated datasets focused on Rett syndrome and Alzheimer's diseas… ▽ More The ever-growing volume of biomedical publications creates a critical need for efficient knowledge discovery. In this context, we introduce an open-source end-to-end framework designed to construct knowledge around specific diseases directly from raw text. To facilitate research in disease-related knowledge discovery, we create two annotated datasets focused on Rett syndrome and Alzheimer's disease, enabling the identification of semantic relations between biomedical entities. Extensive benchmarking explores various ways to represent relations and entity representations, offering insights into optimal modeling strategies for semantic relation detection and highlighting language models' competence in knowledge discovery. We also conduct probing experiments using different layer representations and attention scores to explore transformers' ability to capture semantic relations. △ Less

Submitted 6 September, 2024; v1 submitted 18 July, 2024; originally announced July 2024.

Comments: Under Review

GADePo: Graph-Assisted Declarative Pooling Transformers for Document-Level Relation Extraction

Authors: Andrei C. Coman, Christos Theodoropoulos, Marie-Francine Moens, James Henderson

Abstract: Document-level relation extraction typically relies on text-based encoders and hand-coded pooling heuristics to aggregate information learned by the encoder. In this paper, we leverage the intrinsic graph processing capabilities of the Transformer model and propose replacing hand-coded pooling methods with new tokens in the input, which are designed to aggregate information via explicit graph rela… ▽ More Document-level relation extraction typically relies on text-based encoders and hand-coded pooling heuristics to aggregate information learned by the encoder. In this paper, we leverage the intrinsic graph processing capabilities of the Transformer model and propose replacing hand-coded pooling methods with new tokens in the input, which are designed to aggregate information via explicit graph relations in the computation of attention weights. We introduce a joint text-graph Transformer model and a graph-assisted declarative pooling (GADePo) specification of the input, which provides explicit and high-level instructions for information aggregation. GADePo allows the pooling process to be guided by domain-specific knowledge or desired outcomes but still learned by the Transformer, leading to more flexible and customisable pooling strategies. We evaluate our method across diverse datasets and models and show that our approach yields promising results that are consistently better than those achieved by the hand-coded pooling functions. △ Less

Submitted 6 August, 2024; v1 submitted 28 August, 2023; originally announced August 2023.

Comments: Accepted to KnowledgeNLP workshop at ACL 2024

Representation Learning for Person or Entity-centric Knowledge Graphs: An Application in Healthcare

Authors: Christos Theodoropoulos, Natasha Mulligan, Thaddeus Stappenbeck, Joao Bettencourt-Silva

Abstract: Knowledge graphs (KGs) are a popular way to organise information based on ontologies or schemas and have been used across a variety of scenarios from search to recommendation. Despite advances in KGs, representing knowledge remains a non-trivial task across industries and it is especially challenging in the biomedical and healthcare domains due to complex interdependent relations between entities,… ▽ More Knowledge graphs (KGs) are a popular way to organise information based on ontologies or schemas and have been used across a variety of scenarios from search to recommendation. Despite advances in KGs, representing knowledge remains a non-trivial task across industries and it is especially challenging in the biomedical and healthcare domains due to complex interdependent relations between entities, heterogeneity, lack of standardization, and sparseness of data. KGs are used to discover diagnoses or prioritize genes relevant to disease, but they often rely on schemas that are not centred around a node or entity of interest, such as a person. Entity-centric KGs are relatively unexplored but hold promise in representing important facets connected to a central node and unlocking downstream tasks beyond graph traversal and reasoning, such as generating graph embeddings and training graph neural networks for a wide range of predictive tasks. This paper presents an end-to-end representation learning framework to extract entity-centric KGs from structured and unstructured data. We introduce a star-shaped ontology to represent the multiple facets of a person and use it to guide KG creation. Compact representations of the graphs are created leveraging graph neural networks and experiments are conducted using different levels of heterogeneity or explicitness. A readmission prediction task is used to evaluate the results of the proposed framework, showing a stable system, robust to missing data, that outperforms a range of baseline machine learning classifiers. We highlight that this approach has several potential applications across domains and is open-sourced. Lastly, we discuss lessons learned, challenges, and next steps for the adoption of the framework in practice. △ Less

Submitted 9 October, 2023; v1 submitted 9 May, 2023; originally announced May 2023.

Comments: Accepted into the Twelfth International Conference on Knowledge Capture (K-CAP 2023)

An Information Extraction Study: Take In Mind the Tokenization!

Authors: Christos Theodoropoulos, Marie-Francine Moens

Abstract: Current research on the advantages and trade-offs of using characters, instead of tokenized text, as input for deep learning models, has evolved substantially. New token-free models remove the traditional tokenization step; however, their efficiency remains unclear. Moreover, the effect of tokenization is relatively unexplored in sequence tagging tasks. To this end, we investigate the impact of to… ▽ More Current research on the advantages and trade-offs of using characters, instead of tokenized text, as input for deep learning models, has evolved substantially. New token-free models remove the traditional tokenization step; however, their efficiency remains unclear. Moreover, the effect of tokenization is relatively unexplored in sequence tagging tasks. To this end, we investigate the impact of tokenization when extracting information from documents and present a comparative study and analysis of subword-based and character-based models. Specifically, we study Information Extraction (IE) from biomedical texts. The main outcome is twofold: tokenization patterns can introduce inductive bias that results in state-of-the-art performance, and the character-based models produce promising results; thus, transitioning to token-free IE models is feasible. △ Less

Submitted 1 April, 2023; v1 submitted 27 March, 2023; originally announced March 2023.

Comments: Submitted Manuscript/Preprint (accepted at EUSFLAT 2023, to be published in Lecture Notes in Computer Science (LNCS))

Journal ref: Conference: 2023 13th Conference of the European Society for Fuzzy Logic and Technology (EUSFLAT)

Imposing Relation Structure in Language-Model Embeddings Using Contrastive Learning

Authors: Christos Theodoropoulos, James Henderson, Andrei C. Coman, Marie-Francine Moens

Abstract: Though language model text embeddings have revolutionized NLP research, their ability to capture high-level semantic information, such as relations between entities in text, is limited. In this paper, we propose a novel contrastive learning framework that trains sentence embeddings to encode the relations in a graph structure. Given a sentence (unstructured text) and its graph, we use contrastive… ▽ More Though language model text embeddings have revolutionized NLP research, their ability to capture high-level semantic information, such as relations between entities in text, is limited. In this paper, we propose a novel contrastive learning framework that trains sentence embeddings to encode the relations in a graph structure. Given a sentence (unstructured text) and its graph, we use contrastive learning to impose relation-related structure on the token-level representations of the sentence obtained with a CharacterBERT (El Boukkouri et al.,2020) model. The resulting relation-aware sentence embeddings achieve state-of-the-art results on the relation extraction task using only a simple KNN classifier, thereby demonstrating the success of the proposed method. Additional visualization by a tSNE analysis shows the effectiveness of the learned representation space compared to baselines. Furthermore, we show that we can learn a different space for named entity recognition, again using a contrastive learning objective, and demonstrate how to successfully combine both representation spaces in an entity-relation task. △ Less

Submitted 4 September, 2021; v1 submitted 2 September, 2021; originally announced September 2021.

Comments: To be presented at CoNLL 2021

Journal ref: Conference: 2021 Proceedings of the 25th Conference on Computational Natural Language Learning

Automatic artifact removal of resting-state fMRI with Deep Neural Networks

Authors: Christos Theodoropoulos, Christos Chatzichristos, Sabine Van Huffel

Abstract: Functional Magnetic Resonance Imaging (fMRI) is a non-invasive technique for studying brain activity. During an fMRI session, the subject executes a set of tasks (task-related fMRI study) or no tasks (resting-state fMRI), and a sequence of 3-D brain images is obtained for further analysis. In the course of fMRI, some sources of activation are caused by noise and artifacts. The removal of these sou… ▽ More Functional Magnetic Resonance Imaging (fMRI) is a non-invasive technique for studying brain activity. During an fMRI session, the subject executes a set of tasks (task-related fMRI study) or no tasks (resting-state fMRI), and a sequence of 3-D brain images is obtained for further analysis. In the course of fMRI, some sources of activation are caused by noise and artifacts. The removal of these sources is essential before the analysis of the brain activations. Deep Neural Network (DNN) architectures can be used for denoising and artifact removal. The main advantage of DNN models is the automatic learning of abstract and meaningful features, given the raw data. This work presents advanced DNN architectures for noise and artifact classification, using both spatial and temporal information in resting-state fMRI sessions. The highest performance is achieved by a voting schema using information from all the domains, with an average accuracy of over 98% and a very good balance between the metrics of sensitivity and specificity (98.5% and 97.5% respectively). △ Less

Submitted 7 September, 2021; v1 submitted 15 November, 2020; originally announced November 2020.

Comments: EUSIPCO 2021 (presented)

Journal ref: Conference: 2021 29th European Signal Processing Conference (EUSIPCO)

Input-Output Stability of Barrier-Based Model Predictive Control

Authors: Panagiotis Petsagkourakis, William P. Heath, Joaquin Carrasco, Constantinos Theodoropoulos

Abstract: Conditions for input-output stability of barrier-based model predictive control of linear systems with linear and convex nonlinear (hard or soft) constraints are established through the construction of integral quadratic constraints (IQCs). The IQCs can be used to establish sufficient conditions for global closed-loop stability. In particular conditions for robust stability can be obtained in the… ▽ More Conditions for input-output stability of barrier-based model predictive control of linear systems with linear and convex nonlinear (hard or soft) constraints are established through the construction of integral quadratic constraints (IQCs). The IQCs can be used to establish sufficient conditions for global closed-loop stability. In particular conditions for robust stability can be obtained in the presence of unstructured model uncertainty. IQCs with both static and dynamic multipliers are developed and appropriate convex searches for the multipliers are presented. The effectiveness of the robust stability analysis is demonstrated with an illustrative numerical example. △ Less

Submitted 11 March, 2019; v1 submitted 7 March, 2019; originally announced March 2019.

Stability Analysis of Piecewise Affine Systems with Multi-model Model Predictive Control

Authors: Panagiotis Petsagkourakis, William P. Heath, Constantinos Theodoropoulos

Abstract: Constrained model predictive control (MPC) is a widely used control strategy, which employs moving horizon-based on-line optimisation to compute the optimum path of the manipulated variables. Nonlinear MPC can utilize detailed models but it is computationally expensive; on the other hand linear MPC may not be adequate. Piecewise affine (PWA) models can describe the underlying nonlinear dynamics mo… ▽ More Constrained model predictive control (MPC) is a widely used control strategy, which employs moving horizon-based on-line optimisation to compute the optimum path of the manipulated variables. Nonlinear MPC can utilize detailed models but it is computationally expensive; on the other hand linear MPC may not be adequate. Piecewise affine (PWA) models can describe the underlying nonlinear dynamics more accurately, therefore they can provide a viable trade-off through their use in multi-model linear MPC configurations, which avoid integer programming. However, such schemes may introduce uncertainty affecting the closed loop stability. In this work, we propose an input to output stability analysis for closed loop systems, consisting of PWA models, where an observer and multi-model linear MPC are applied together, under unstructured uncertainty. Integral quadratic constraints (IQCs) are employed to assess the robustness of MPC under uncertainty. We create a model pool, by performing linearisation on selected transient points. All the possible uncertainties and nonlinearities (including the controller) can be introduced in the framework, assuming that they admit the appropriate IQCs, whilst the dissipation inequality can provide necessary conditions incorporating IQCs. We demonstrate the existence of static multipliers, which can reduce the conservatism of the stability analysis significantly. The proposed methodology is demonstrated through two engineering case studies. △ Less

Submitted 1 August, 2018; originally announced August 2018.

Comments: 28 pages 9 figures

Equation-Free Multiscale Computation: enabling microscopic simulators to perform system-level tasks

Authors: Ioannis G. Kevrekidis, C. William Gear, James M. Hyman, Panagiotis G. Kevrekidis, Olof Runborg, Constantinos Theodoropoulos

Abstract: We present and discuss a framework for computer-aided multiscale analysis, which enables models at a "fine" (microscopic/stochastic) level of description to perform modeling tasks at a "coarse" (macroscopic, systems) level. These macroscopic modeling tasks, yielding information over long time and large space scales, are accomplished through appropriately initialized calls to the microscopic simu… ▽ More We present and discuss a framework for computer-aided multiscale analysis, which enables models at a "fine" (microscopic/stochastic) level of description to perform modeling tasks at a "coarse" (macroscopic, systems) level. These macroscopic modeling tasks, yielding information over long time and large space scales, are accomplished through appropriately initialized calls to the microscopic simulator for only short times and small spatial domains. Our equation-free (EF) approach, when successful, can bypass the derivation of the macroscopic evolution equations when these equations conceptually exist but are not available in closed form. We discuss how the mathematics-assisted development of a computational superstructure may enable alternative descriptions of the problem physics (e.g. Lattice Boltzmann (LB), kinetic Monte Carlo (KMC) or Molecular Dynamics (MD) microscopic simulators, executed over relatively short time and space scales) to perform systems level tasks (integration over relatively large time and space scales,"coarse" bifurcation analysis, optimization, and control) directly. In effect, the procedure constitutes a systems identification based, "closure on demand" computational toolkit, bridging microscopic/stochastic simulation with traditional continuum scientific computation and numerical analysis. We illustrate these ideas through examples from chemical kinetics (LB, KMC), rheology (Brownian Dynamics), homogenization and the computation of "coarsely self-similar" solutions, and discuss various features, limitations and potential extensions of the approach. △ Less

Submitted 10 September, 2002; originally announced September 2002.

Coarse Bifurcation Studies of Bubble Flow Microscopic Simulations

Authors: C. Theodoropoulos, K. Sankaranarayanan, S. Sundaresan, I. G. Kevrekidis

Abstract: The parametric behavior of regular periodic arrays of rising bubbles is investigated with the aid of 2-dimensional BGK Lattice-Boltzmann (LB) simulators. The Recursive Projection Method is implemented and coupled to the LB simulators, accelerating their convergence towards what we term coarse steady states. Efficient stability/bifurcation analysis is performed by computing the leading eigenvalue… ▽ More The parametric behavior of regular periodic arrays of rising bubbles is investigated with the aid of 2-dimensional BGK Lattice-Boltzmann (LB) simulators. The Recursive Projection Method is implemented and coupled to the LB simulators, accelerating their convergence towards what we term coarse steady states. Efficient stability/bifurcation analysis is performed by computing the leading eigenvalues/eigenvectors of the coarse time stepper. Our approach constitutes the basis for system-level analysis of processes modeled through microscopic simulations. △ Less

Submitted 16 November, 2001; originally announced November 2001.

Comments: 4 pages, 3 figures