Showing 1–40 of 40 results for author: Gómez, V

Hierarchical Average-Reward Linearly-solvable Markov Decision Processes

Authors: Guillermo Infante, Anders Jonsson, Vicenç Gómez

Abstract: We introduce a novel approach to hierarchical reinforcement learning for Linearly-solvable Markov Decision Processes (LMDPs) in the infinite-horizon average-reward setting. Unlike previous work, our approach allows learning low-level and high-level tasks simultaneously, without imposing limiting restrictions on the low-level tasks. Our method relies on partitions of the state space that create sma… ▽ More We introduce a novel approach to hierarchical reinforcement learning for Linearly-solvable Markov Decision Processes (LMDPs) in the infinite-horizon average-reward setting. Unlike previous work, our approach allows learning low-level and high-level tasks simultaneously, without imposing limiting restrictions on the low-level tasks. Our method relies on partitions of the state space that create smaller subtasks that are easier to solve, and the equivalence between such partitions to learn more efficiently. We then exploit the compositionality of low-level tasks to exactly represent the value function of the high-level task. Experiments show that our approach can outperform flat average-reward reinforcement learning by one or several orders of magnitude. △ Less

Submitted 9 July, 2024; originally announced July 2024.

Combined Task and Motion Planning Via Sketch Decompositions (Extended Version with Supplementary Material)

Authors: Magí Dalmau-Moreno, Néstor García, Vicenç Gómez, Héctor Geffner

Abstract: The challenge in combined task and motion planning (TAMP) is the effective integration of a search over a combinatorial space, usually carried out by a task planner, and a search over a continuous configuration space, carried out by a motion planner. Using motion planners for testing the feasibility of task plans and filling out the details is not effective because it makes the geometrical constra… ▽ More The challenge in combined task and motion planning (TAMP) is the effective integration of a search over a combinatorial space, usually carried out by a task planner, and a search over a continuous configuration space, carried out by a motion planner. Using motion planners for testing the feasibility of task plans and filling out the details is not effective because it makes the geometrical constraints play a passive role. This work introduces a new interleaved approach for integrating the two dimensions of TAMP that makes use of sketches, a recent simple but powerful language for expressing the decomposition of problems into subproblems. A sketch has width 1 if it decomposes the problem into subproblems that can be solved greedily in linear time. In the paper, a general sketch is introduced for several classes of TAMP problems which has width 1 under suitable assumptions. While sketch decompositions have been developed for classical planning, they offer two important benefits in the context of TAMP. First, when a task plan is found to be unfeasible due to the geometric constraints, the combinatorial search resumes in a specific sub-problem. Second, the sampling of object configurations is not done once, globally, at the start of the search, but locally, at the start of each subproblem. Optimizations of this basic setting are also considered and experimental results over existing and new pick-and-place benchmarks are reported. △ Less

Submitted 24 March, 2024; originally announced March 2024.

Planning with a Learned Policy Basis to Optimally Solve Complex Tasks

Authors: Guillermo Infante, David Kuric, Anders Jonsson, Vicenç Gómez, Herke van Hoof

Abstract: Conventional reinforcement learning (RL) methods can successfully solve a wide range of sequential decision problems. However, learning policies that can generalize predictably across multiple tasks in a setting with non-Markovian reward specifications is a challenging problem. We propose to use successor features to learn a policy basis so that each (sub)policy in it solves a well-defined subprob… ▽ More Conventional reinforcement learning (RL) methods can successfully solve a wide range of sequential decision problems. However, learning policies that can generalize predictably across multiple tasks in a setting with non-Markovian reward specifications is a challenging problem. We propose to use successor features to learn a policy basis so that each (sub)policy in it solves a well-defined subproblem. In a task described by a finite state automaton (FSA) that involves the same set of subproblems, the combination of these (sub)policies can then be used to generate an optimal solution without additional learning. In contrast to other methods that combine (sub)policies via planning, our method asymptotically attains global optimality, even in stochastic environments. △ Less

Submitted 3 June, 2024; v1 submitted 22 March, 2024; originally announced March 2024.

Improving Subgraph-GNNs via Edge-Level Ego-Network Encodings

Authors: Nurudin Alvarez-Gonzalez, Andreas Kaltenbrunner, Vicenç Gómez

Abstract: We present a novel edge-level ego-network encoding for learning on graphs that can boost Message Passing Graph Neural Networks (MP-GNNs) by providing additional node and edge features or extending message-passing formats. The proposed encoding is sufficient to distinguish Strongly Regular Graphs, a family of challenging 3-WL equivalent graphs. We show theoretically that such encoding is more expre… ▽ More We present a novel edge-level ego-network encoding for learning on graphs that can boost Message Passing Graph Neural Networks (MP-GNNs) by providing additional node and edge features or extending message-passing formats. The proposed encoding is sufficient to distinguish Strongly Regular Graphs, a family of challenging 3-WL equivalent graphs. We show theoretically that such encoding is more expressive than node-based sub-graph MP-GNNs. In an empirical evaluation on four benchmarks with 10 graph datasets, our results match or improve previous baselines on expressivity, graph classification, graph regression, and proximity tasks -- while reducing memory usage by 18.1x in certain real-world settings. △ Less

Submitted 2 May, 2024; v1 submitted 10 December, 2023; originally announced December 2023.

Comments: TMLR, graph neural networks, weisfeiler-lehman, expressivity, higher-order GNNs, 3-WL, 1-WL, edge-level, ego-networks

Journal ref: Nurudin Alvarez-Gonzalez, Andreas Kaltenbrunner, Vicenç Gomez. Improving Subgraph-GNNs via Edge-Level Ego-Network Encodings. In Transactions on Machine Learning Research, 2024

Unpacking polarization: Antagonism and Alignment in Signed Networks of Online Interaction

Authors: Emma Fraxanet, Max Pellert, Simon Schweighofer, Vicenç Gómez, David Garcia

Abstract: Political conflict is an essential element of democratic systems, but can also threaten their existence if it becomes too intense. This happens particularly when most political issues become aligned along the same major fault line, splitting society into two antagonistic camps. In the 20th century, major fault lines were formed by structural conflicts, like owners vs workers, center vs periphery,… ▽ More Political conflict is an essential element of democratic systems, but can also threaten their existence if it becomes too intense. This happens particularly when most political issues become aligned along the same major fault line, splitting society into two antagonistic camps. In the 20th century, major fault lines were formed by structural conflicts, like owners vs workers, center vs periphery, etc. But these classical cleavages have since lost their explanatory power. Instead of theorizing new cleavages, we present the FAULTANA (FAULT-line Alignment Network Analysis) pipeline, a computational method to uncover major fault lines in data of signed online interactions. Our method makes it possible to quantify the degree of antagonism prevalent in different online debates, as well as how aligned each debate is to the major fault line. This makes it possible to identify the wedge issues driving polarization, characterized by both intense antagonism and alignment. We apply our approach to large-scale data sets of Birdwatch, a US-based Twitter fact-checking community and the discussion forums of DerStandard, an Austrian online newspaper. We find that both online communities are divided into two large groups and that their separation follows political identities and topics. In addition, for DerStandard, we pinpoint issues that reinforce societal fault lines and thus drive polarization. We also identify issues that trigger online conflict without strictly aligning with those dividing lines (e.g. COVID-19). Our methods allow us to construct a time-resolved picture of affective polarization that shows the separate contributions of cohesiveness and divisiveness to the dynamics of alignment during contentious elections and events. △ Less

Submitted 2 February, 2024; v1 submitted 13 July, 2023; originally announced July 2023.

Category Theory for Autonomous Robots: The Marathon 2 Use Case

Authors: Esther Aguado, Virgilio Gómez, Miguel Hernando, Claudio Rossi, Ricardo Sanz

Abstract: Model-based systems engineering (MBSE) is a methodology that exploits system representation during the entire system life-cycle. The use of formal models has gained momentum in robotics engineering over the past few years. Models play a crucial role in robot design; they serve as the basis for achieving holistic properties, such as functional reliability or adaptive resilience, and facilitate the… ▽ More Model-based systems engineering (MBSE) is a methodology that exploits system representation during the entire system life-cycle. The use of formal models has gained momentum in robotics engineering over the past few years. Models play a crucial role in robot design; they serve as the basis for achieving holistic properties, such as functional reliability or adaptive resilience, and facilitate the automated production of modules. We propose the use of formal conceptualizations beyond the engineering phase, providing accurate models that can be leveraged at runtime. This paper explores the use of Category Theory, a mathematical framework for describing abstractions, as a formal language to produce such robot models. To showcase its practical application, we present a concrete example based on the Marathon 2 experiment. Here, we illustrate the potential of formalizing systems -- including their recovery mechanisms -- which allows engineers to design more trustworthy autonomous robots. This, in turn, enhances their dependability and performance. △ Less

Submitted 18 September, 2023; v1 submitted 2 March, 2023; originally announced March 2023.

Beyond 1-WL with Local Ego-Network Encodings

Authors: Nurudin Alvarez-Gonzalez, Andreas Kaltenbrunner, Vicenç Gómez

Abstract: Identifying similar network structures is key to capture graph isomorphisms and learn representations that exploit structural information encoded in graph data. This work shows that ego-networks can produce a structural encoding scheme for arbitrary graphs with greater expressivity than the Weisfeiler-Lehman (1-WL) test. We introduce IGEL, a preprocessing step to produce features that augment node… ▽ More Identifying similar network structures is key to capture graph isomorphisms and learn representations that exploit structural information encoded in graph data. This work shows that ego-networks can produce a structural encoding scheme for arbitrary graphs with greater expressivity than the Weisfeiler-Lehman (1-WL) test. We introduce IGEL, a preprocessing step to produce features that augment node representations by encoding ego-networks into sparse vectors that enrich Message Passing (MP) Graph Neural Networks (GNNs) beyond 1-WL expressivity. We describe formally the relation between IGEL and 1-WL, and characterize its expressive power and limitations. Experiments show that IGEL matches the empirical expressivity of state-of-the-art methods on isomorphism detection while improving performance on seven GNN architectures. △ Less

Submitted 7 December, 2022; v1 submitted 27 November, 2022; originally announced November 2022.

Comments: Presented at the First Learning on Graphs Conference (LoG 2022), Virtual Event, December 9-12, 2022

Crowding out the truth? A simple model of misinformation, polarization and meaningful social interactions

Authors: Fabrizio Germano, Vicenç Gómez, Francesco Sobbrio

Abstract: This paper provides a simple theoretical framework to evaluate the effect of key parameters of ranking algorithms, namely popularity and personalization parameters, on measures of platform engagement, misinformation and polarization. The results show that an increase in the weight assigned to online social interactions (e.g., likes and shares) and to personalized content may increase engagement on… ▽ More This paper provides a simple theoretical framework to evaluate the effect of key parameters of ranking algorithms, namely popularity and personalization parameters, on measures of platform engagement, misinformation and polarization. The results show that an increase in the weight assigned to online social interactions (e.g., likes and shares) and to personalized content may increase engagement on the social media platform, while at the same time increasing misinformation and/or polarization. By exploiting Facebook's 2018 "Meaningful Social Interactions" algorithmic ranking update, we also provide direct empirical support for some of the main predictions of the model. △ Less

Submitted 5 October, 2022; originally announced October 2022.

Comments: 21 pages + appendices, 11 figures, preprint

MSC Class: 91C99 ACM Class: J.4

Uncovering the Limits of Text-based Emotion Detection

Authors: Nurudin Alvarez-Gonzalez, Andreas Kaltenbrunner, Vicenç Gómez

Abstract: Identifying emotions from text is crucial for a variety of real world tasks. We consider the two largest now-available corpora for emotion classification: GoEmotions, with 58k messages labelled by readers, and Vent, with 33M writer-labelled messages. We design a benchmark and evaluate several feature spaces and learning algorithms, including two simple yet novel models on top of BERT that outperfo… ▽ More Identifying emotions from text is crucial for a variety of real world tasks. We consider the two largest now-available corpora for emotion classification: GoEmotions, with 58k messages labelled by readers, and Vent, with 33M writer-labelled messages. We design a benchmark and evaluate several feature spaces and learning algorithms, including two simple yet novel models on top of BERT that outperform previous strong baselines on GoEmotions. Through an experiment with human participants, we also analyze the differences between how writers express emotions and how readers perceive them. Our results suggest that emotions expressed by writers are harder to identify than emotions that readers perceive. We share a public web interface for researchers to explore our models. △ Less

Submitted 30 October, 2021; v1 submitted 4 September, 2021; originally announced September 2021.

Comments: Accepted for publication in Findings of EMNLP 2021

Globally Optimal Hierarchical Reinforcement Learning for Linearly-Solvable Markov Decision Processes

Authors: Guillermo Infante, Anders Jonsson, Vicenç Gómez

Abstract: In this work we present a novel approach to hierarchical reinforcement learning for linearly-solvable Markov decision processes. Our approach assumes that the state space is partitioned, and the subtasks consist in moving between the partitions. We represent value functions on several levels of abstraction, and use the compositionality of subtasks to estimate the optimal values of the states in ea… ▽ More In this work we present a novel approach to hierarchical reinforcement learning for linearly-solvable Markov decision processes. Our approach assumes that the state space is partitioned, and the subtasks consist in moving between the partitions. We represent value functions on several levels of abstraction, and use the compositionality of subtasks to estimate the optimal values of the states in each partition. The policy is implicitly defined on these optimal value estimates, rather than being decomposed among the subtasks. As a consequence, our approach can learn the globally optimal policy, and does not suffer from the non-stationarity of high-level decisions. If several partitions have equivalent dynamics, the subtasks of those partitions can be shared. If the set of boundary states is smaller than the entire state space, our approach can have significantly smaller sample complexity than that of a flat learner, and we validate this empirically in several experiments. △ Less

Submitted 28 April, 2022; v1 submitted 29 June, 2021; originally announced June 2021.

Journal ref: The Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22), 2022

Hierarchical Width-Based Planning and Learning

Authors: Miquel Junyent, Vicenç Gómez, Anders Jonsson

Abstract: Width-based search methods have demonstrated state-of-the-art performance in a wide range of testbeds, from classical planning problems to image-based simulators such as Atari games. These methods scale independently of the size of the state-space, but exponentially in the problem width. In practice, running the algorithm with a width larger than 1 is computationally intractable, prohibiting IW fr… ▽ More Width-based search methods have demonstrated state-of-the-art performance in a wide range of testbeds, from classical planning problems to image-based simulators such as Atari games. These methods scale independently of the size of the state-space, but exponentially in the problem width. In practice, running the algorithm with a width larger than 1 is computationally intractable, prohibiting IW from solving higher width problems. In this paper, we present a hierarchical algorithm that plans at two levels of abstraction. A high-level planner uses abstract features that are incrementally discovered from low-level pruning decisions. We illustrate this algorithm in classical planning PDDL domains as well as in pixel-based simulator domains. In classical planning, we show how IW(1) at two levels of abstraction can solve problems of width 2. For pixel-based domains, we show how in combination with a learned policy and a learned value function, the proposed hierarchical IW can outperform current flat IW-based planners in Atari games with sparse rewards. △ Less

Submitted 1 September, 2021; v1 submitted 15 January, 2021; originally announced January 2021.

Journal ref: Proceedings of the Thirty-First International Conference on Automated Planning and Scheduling (ICAPS 2021)

Inductive Graph Embeddings through Locality Encodings

Authors: Nurudin Alvarez-Gonzalez, Andreas Kaltenbrunner, Vicenç Gómez

Abstract: Learning embeddings from large-scale networks is an open challenge. Despite the overwhelming number of existing methods, is is unclear how to exploit network structure in a way that generalizes easily to unseen nodes, edges or graphs. In this work, we look at the problem of finding inductive network embeddings in large networks without domain-dependent node/edge attributes. We propose to use a set… ▽ More Learning embeddings from large-scale networks is an open challenge. Despite the overwhelming number of existing methods, is is unclear how to exploit network structure in a way that generalizes easily to unseen nodes, edges or graphs. In this work, we look at the problem of finding inductive network embeddings in large networks without domain-dependent node/edge attributes. We propose to use a set of basic predefined local encodings as the basis of a learning algorithm. In particular, we consider the degree frequencies at different distances from a node, which can be computed efficiently for relatively short distances and a large number of nodes. Interestingly, the resulting embeddings generalize well across unseen or distant regions in the network, both in unsupervised settings, when combined with language model learning, as well as in supervised tasks, when used as additional features in a neural network. Despite its simplicity, this method achieves state-of-the-art performance in tasks such as role detection, link prediction and node classification, and represents an inductive network embedding method directly applicable to large unattributed networks. △ Less

Submitted 26 September, 2020; originally announced September 2020.

Adaptive Smoothing Path Integral Control

Authors: Dominik Thalmeier, Hilbert J. Kappen, Simone Totaro, Vicenç Gómez

Abstract: In Path Integral control problems a representation of an optimally controlled dynamical system can be formally computed and serve as a guidepost to learn a parametrized policy. The Path Integral Cross-Entropy (PICE) method tries to exploit this, but is hampered by poor sample efficiency. We propose a model-free algorithm called ASPIC (Adaptive Smoothing of Path Integral Control) that applies an in… ▽ More In Path Integral control problems a representation of an optimally controlled dynamical system can be formally computed and serve as a guidepost to learn a parametrized policy. The Path Integral Cross-Entropy (PICE) method tries to exploit this, but is hampered by poor sample efficiency. We propose a model-free algorithm called ASPIC (Adaptive Smoothing of Path Integral Control) that applies an inf-convolution to the cost function to speedup convergence of policy optimization. We identify PICE as the infinite smoothing limit of such technique and show that the sample efficiency problems that PICE suffers disappear for finite levels of smoothing. For zero smoothing this method becomes a greedy optimization of the cost, which is the standard approach in current reinforcement learning. We show analytically and empirically that intermediate levels of smoothing are optimal, which renders the new method superior to both PICE and direct cost-optimization. △ Less

Submitted 13 May, 2020; originally announced May 2020.

Comments: 23 pages, 5 figures, NeurIPS 2019 Optimization Foundations of Reinforcement Learning Workshop (OptRL 2019)

Input complexity and out-of-distribution detection with likelihood-based generative models

Authors: Joan Serrà, David Álvarez, Vicenç Gómez, Olga Slizovskaia, José F. Núñez, Jordi Luque

Abstract: Likelihood-based generative models are a promising resource to detect out-of-distribution (OOD) inputs which could compromise the robustness or reliability of a machine learning system. However, likelihoods derived from such models have been shown to be problematic for detecting certain types of inputs that significantly differ from training data. In this paper, we pose that this problem is due to… ▽ More Likelihood-based generative models are a promising resource to detect out-of-distribution (OOD) inputs which could compromise the robustness or reliability of a machine learning system. However, likelihoods derived from such models have been shown to be problematic for detecting certain types of inputs that significantly differ from training data. In this paper, we pose that this problem is due to the excessive influence that input complexity has in generative models' likelihoods. We report a set of experiments supporting this hypothesis, and use an estimate of input complexity to derive an efficient and parameter-free OOD score, which can be seen as a likelihood-ratio, akin to Bayesian model comparison. We find such score to perform comparably to, or even better than, existing OOD detection approaches under a wide range of data sets, models, model sizes, and complexity estimates. △ Less

Submitted 17 January, 2020; v1 submitted 25 September, 2019; originally announced September 2019.

Comments: Accepted for ICLR2020

Consequential Ranking Algorithms and Long-term Welfare

Authors: Behzad Tabibian, Vicenç Gómez, Abir De, Bernhard Schölkopf, Manuel Gomez Rodriguez

Abstract: Ranking models are typically designed to provide rankings that optimize some measure of immediate utility to the users. As a result, they have been unable to anticipate an increasing number of undesirable long-term consequences of their proposed rankings, from fueling the spread of misinformation and increasing polarization to degrading social discourse. Can we design ranking models that understan… ▽ More Ranking models are typically designed to provide rankings that optimize some measure of immediate utility to the users. As a result, they have been unable to anticipate an increasing number of undesirable long-term consequences of their proposed rankings, from fueling the spread of misinformation and increasing polarization to degrading social discourse. Can we design ranking models that understand the consequences of their proposed rankings and, more importantly, are able to avoid the undesirable ones? In this paper, we first introduce a joint representation of rankings and user dynamics using Markov decision processes. Then, we show that this representation greatly simplifies the construction of consequential ranking models that trade off the immediate utility and the long-term welfare. In particular, we can obtain optimal consequential rankings just by applying weighted sampling on the rankings provided by models that maximize measures of immediate utility. However, in practice, such a strategy may be inefficient and impractical, specially in high dimensional scenarios. To overcome this, we introduce an efficient gradient-based algorithm to learn parameterized consequential ranking models that effectively approximate optimal ones. We showcase our methodology using synthetic and real data gathered from Reddit and show that ranking models derived using our methodology provide ranks that may mitigate the spread of misinformation and improve the civility of online discussions. △ Less

Submitted 13 May, 2019; originally announced May 2019.

Deep Policies for Width-Based Planning in Pixel Domains

Authors: Miquel Junyent, Anders Jonsson, Vicenç Gómez

Abstract: Width-based planning has demonstrated great success in recent years due to its ability to scale independently of the size of the state space. For example, Bandres et al. (2018) introduced a rollout version of the Iterated Width algorithm whose performance compares well with humans and learning methods in the pixel setting of the Atari games suite. In this setting, planning is done on-line using th… ▽ More Width-based planning has demonstrated great success in recent years due to its ability to scale independently of the size of the state space. For example, Bandres et al. (2018) introduced a rollout version of the Iterated Width algorithm whose performance compares well with humans and learning methods in the pixel setting of the Atari games suite. In this setting, planning is done on-line using the "screen" states and selecting actions by looking ahead into the future. However, this algorithm is purely exploratory and does not leverage past reward information. Furthermore, it requires the state to be factored into features that need to be pre-defined for the particular task, e.g., the B-PROST pixel features. In this work, we extend width-based planning by incorporating an explicit policy in the action selection mechanism. Our method, called $π$-IW, interleaves width-based planning and policy learning using the state-actions visited by the planner. The policy estimate takes the form of a neural network and is in turn used to guide the planning step, thus reinforcing promising paths. Surprisingly, we observe that the representation learned by the neural network can be used as a feature space for the width-based planner without degrading its performance, thus removing the requirement of pre-defined features for the planner. We compare $π$-IW with previous width-based methods and with AlphaZero, a method that also interleaves planning and learning, in simple environments, and show that $π$-IW has superior performance. We also show that $π$-IW algorithm outperforms previous width-based methods in the pixel setting of Atari games suite. △ Less

Submitted 5 October, 2021; v1 submitted 12 April, 2019; originally announced April 2019.

Comments: In Proceedings of the 29th International Conference on Automated Planning and Scheduling (ICAPS 2019). arXiv admin note: text overlap with arXiv:1806.05898

The few-get-richer: a surprising consequence of popularity-based rankings

Authors: Fabrizio Germano, Vicenç Gómez, Gaël Le Mens

Abstract: Ranking algorithms play a crucial role in online platforms ranging from search engines to recommender systems. In this paper, we identify a surprising consequence of popularity-based rankings: the fewer the items reporting a given signal, the higher the share of the overall traffic they collectively attract. This few-get-richer effect emerges in settings where there are few distinct classes of ite… ▽ More Ranking algorithms play a crucial role in online platforms ranging from search engines to recommender systems. In this paper, we identify a surprising consequence of popularity-based rankings: the fewer the items reporting a given signal, the higher the share of the overall traffic they collectively attract. This few-get-richer effect emerges in settings where there are few distinct classes of items (e.g., left-leaning news sources versus right-leaning news sources), and items are ranked based on their popularity. We demonstrate analytically that the few-get-richer effect emerges when people tend to click on top-ranked items and have heterogeneous preferences for the classes of items. Using simulations, we analyze how the strength of the effect changes with assumptions about the setting and human behavior. We also test our predictions experimentally in an online experiment with human participants. Our findings have important implications to understand the spread of misinformation. △ Less

Submitted 14 June, 2019; v1 submitted 7 February, 2019; originally announced February 2019.

Comments: 7 pages, 3 figures, 1 table, Proceedings of The Web Conference (WWW 2019)

ACM Class: H.1; H.3.3; G.3

Journal ref: WWW '19: The World Wide Web ConferenceMay 2019

Sharing emotions at scale: The Vent dataset

Authors: Nikolaos Lykousas, Costantinos Patsakis, Andreas Kaltenbrunner, Vicenç Gómez

Abstract: The continuous and increasing use of social media has enabled the expression of human thoughts, opinions, and everyday actions publicly at an unprecedented scale. We present the Vent dataset, the largest annotated dataset of text, emotions, and social connections to date. It comprises more than 33 millions of posts by nearly a million of users together with their social connections. Each post has… ▽ More The continuous and increasing use of social media has enabled the expression of human thoughts, opinions, and everyday actions publicly at an unprecedented scale. We present the Vent dataset, the largest annotated dataset of text, emotions, and social connections to date. It comprises more than 33 millions of posts by nearly a million of users together with their social connections. Each post has an associated emotion. There are 705 different emotions, organized in 63 "emotion categories", forming a two-level taxonomy of affects. Our initial statistical analysis describes the global patterns of activity in the Vent platform, revealing large heterogenities and certain remarkable regularities regarding the use of the different emotions. We focus on the aggregated use of emotions, the temporal activity, and the social network of users, and outline possible methods to infer emotion networks based on the user activity. We also analyze the text and describe the affective landscape of Vent, finding agreements with existing (small scale) annotated corpus in terms of emotion categories and positive/negative valences. Finally, we discuss possible research questions that can be addressed from this unique dataset. △ Less

Submitted 24 March, 2019; v1 submitted 15 January, 2019; originally announced January 2019.

Comments: 9 pages, 12 figures, 2 tables. Accepted at the 13th International AAAI Conference on Web and Social Media (ICWSM 2019)

Interactive Discovery System for Direct Democracy

Authors: Pablo Aragón, Yago Bermejo, Vicenç Gómez, Andreas Kaltenbrunner

Abstract: Decide Madrid is the civic technology of Madrid City Council which allows users to create and support online petitions. Despite the initial success, the platform is encountering problems with the growth of petition signing because petitions are far from the minimum number of supporting votes they must gather. Previous analyses have suggested that this problem is produced by the interface: a pagina… ▽ More Decide Madrid is the civic technology of Madrid City Council which allows users to create and support online petitions. Despite the initial success, the platform is encountering problems with the growth of petition signing because petitions are far from the minimum number of supporting votes they must gather. Previous analyses have suggested that this problem is produced by the interface: a paginated list of petitions which applies a non-optimal ranking algorithm. For this reason, we present an interactive system for the discovery of topics and petitions. This approach leads us to reflect on the usefulness of data visualization techniques to address relevant societal challenges. △ Less

Submitted 12 July, 2018; originally announced July 2018.

Comments: Accepted at the 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 18). For academic purposes, please cite the conference version

Adult content in Social Live Streaming Services: Characterizing deviant users and relationships

Authors: Nikolaos Lykousas, Vicenç Gómez, Constantinos Patsakis

Abstract: Social Live Stream Services (SLSS) exploit a new level of social interaction. One of the main challenges in these services is how to detect and prevent deviant behaviors that violate community guidelines. In this work, we focus on adult content production and consumption in two widely used SLSS, namely Live.me and Loops Live, which have millions of users producing massive amounts of video content… ▽ More Social Live Stream Services (SLSS) exploit a new level of social interaction. One of the main challenges in these services is how to detect and prevent deviant behaviors that violate community guidelines. In this work, we focus on adult content production and consumption in two widely used SLSS, namely Live.me and Loops Live, which have millions of users producing massive amounts of video content on a daily basis. We use a pre-trained deep learning model to identify broadcasters of adult content. Our results indicate that moderation systems in place are highly ineffective in suspending the accounts of such users. We create two large datasets by crawling the social graphs of these platforms, which we analyze to identify characterizing traits of adult content producers and consumers, and discover interesting patterns of relationships among them, evident in both networks. △ Less

Submitted 27 June, 2018; originally announced June 2018.

Comments: 8 pages, 4 figures, 5 tables. IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining

Online Petitioning Through Data Exploration and What We Found There: A Dataset of Petitions from Avaaz.org

Authors: Pablo Aragón, Diego Sáez-Trumper, Miriam Redi, Scott A. Hale, Vicenç Gómez, Andreas Kaltenbrunner

Abstract: The Internet has become a fundamental resource for activism as it facilitates political mobilization at a global scale. Petition platforms are a clear example of how thousands of people around the world can contribute to social change. Avaaz.org, with a presence in over 200 countries, is one of the most popular of this type. However, little research has focused on this platform, probably due to a… ▽ More The Internet has become a fundamental resource for activism as it facilitates political mobilization at a global scale. Petition platforms are a clear example of how thousands of people around the world can contribute to social change. Avaaz.org, with a presence in over 200 countries, is one of the most popular of this type. However, little research has focused on this platform, probably due to a lack of available data. In this work we retrieved more than 350K petitions, standardized their field values, and added new information using language detection and named-entity recognition. To motivate future research with this unique repository of global protest, we present a first exploration of the dataset. In particular, we examine how social media campaigning is related to the success of petitions, as well as some geographic and linguistic findings about the worldwide community of Avaaz.org. We conclude with example research questions that could be addressed with our dataset. △ Less

Submitted 21 June, 2018; originally announced June 2018.

Comments: Accepted as a dataset paper at the 12th International AAAI Conference on Web and Social Media (ICWSM-18). This preprint includes an additional appendix with the reasons, provided by Avaaz.org, about the anomalies detected when exploring the dataset. For academic purposes, please cite the ICWSM version

Improving width-based planning with compact policies

Authors: Miquel Junyent, Anders Jonsson, Vicenç Gómez

Abstract: Optimal action selection in decision problems characterized by sparse, delayed rewards is still an open challenge. For these problems, current deep reinforcement learning methods require enormous amounts of data to learn controllers that reach human-level performance. In this work, we propose a method that interleaves planning and learning to address this issue. The planning step hinges on the Ite… ▽ More Optimal action selection in decision problems characterized by sparse, delayed rewards is still an open challenge. For these problems, current deep reinforcement learning methods require enormous amounts of data to learn controllers that reach human-level performance. In this work, we propose a method that interleaves planning and learning to address this issue. The planning step hinges on the Iterated-Width (IW) planner, a state of the art planner that makes explicit use of the state representation to perform structured exploration. IW is able to scale up to problems independently of the size of the state space. From the state-actions visited by IW, the learning step estimates a compact policy, which in turn is used to guide the planning step. The type of exploration used by our method is radically different than the standard random exploration used in RL. We evaluate our method in simple problems where we show it to have superior performance than the state-of-the-art reinforcement learning algorithms A2C and Alpha Zero. Finally, we present preliminary results in a subset of the Atari games suite. △ Less

Submitted 15 June, 2018; originally announced June 2018.

Deliberative Platform Design: The case study of the online discussions in Decidim Barcelona

Authors: Pablo Aragón, Andreas Kaltenbrunner, Antonio Calleja-López, Andrés Pereira, Arnau Monterde, Xabier E. Barandiaran, Vicenç Gómez

Abstract: With the irruption of ICTs and the crisis of political representation, many online platforms have been developed with the aim of improving participatory democratic processes. However, regarding platforms for online petitioning, previous research has not found examples of how to effectively introduce discussions, a crucial feature to promote deliberation. In this study we focus on the case of Decid… ▽ More With the irruption of ICTs and the crisis of political representation, many online platforms have been developed with the aim of improving participatory democratic processes. However, regarding platforms for online petitioning, previous research has not found examples of how to effectively introduce discussions, a crucial feature to promote deliberation. In this study we focus on the case of Decidim Barcelona, the online participatory-democracy platform launched by the City Council of Barcelona in which proposals can be discussed with an interface that combines threaded discussions and comment alignment with the proposal. This innovative approach allows to examine whether neutral, positive or negative comments are more likely to generate discussion cascades. The results reveal that, with this interface, comments marked as negatively aligned with the proposal were more likely to engage users in online discussions and, therefore, helped to promote deliberative decision making. △ Less

Submitted 20 July, 2017; originally announced July 2017.

Comments: Accepted at the 9th International Conference on Social Informatics (SocInfo 2017)

A unified view of entropy-regularized Markov decision processes

Authors: Gergely Neu, Anders Jonsson, Vicenç Gómez

Abstract: We propose a general framework for entropy-regularized average-reward reinforcement learning in Markov decision processes (MDPs). Our approach is based on extending the linear-programming formulation of policy optimization in MDPs to accommodate convex regularization functions. Our key result is showing that using the conditional entropy of the joint state-action distributions as regularization yi… ▽ More We propose a general framework for entropy-regularized average-reward reinforcement learning in Markov decision processes (MDPs). Our approach is based on extending the linear-programming formulation of policy optimization in MDPs to accommodate convex regularization functions. Our key result is showing that using the conditional entropy of the joint state-action distributions as regularization yields a dual optimization problem closely resembling the Bellman optimality equations. This result enables us to formalize a number of state-of-the-art entropy-regularized reinforcement learning algorithms as approximate variants of Mirror Descent or Dual Averaging, and thus to argue about the convergence properties of these methods. In particular, we show that the exact version of the TRPO algorithm of Schulman et al. (2015) actually converges to the optimal policy, while the entropy-regularized policy gradient methods of Mnih et al. (2016) may fail to converge to a fixed point. Finally, we illustrate empirically the effects of using various regularization techniques on learning performance in a simple reinforcement learning setup. △ Less

Submitted 22 May, 2017; originally announced May 2017.

Fast rates for online learning in Linearly Solvable Markov Decision Processes

Authors: Gergely Neu, Vicenç Gómez

Abstract: We study the problem of online learning in a class of Markov decision processes known as linearly solvable MDPs. In the stationary version of this problem, a learner interacts with its environment by directly controlling the state transitions, attempting to balance a fixed state-dependent cost and a certain smooth cost penalizing extreme control inputs. In the current paper, we consider an online… ▽ More We study the problem of online learning in a class of Markov decision processes known as linearly solvable MDPs. In the stationary version of this problem, a learner interacts with its environment by directly controlling the state transitions, attempting to balance a fixed state-dependent cost and a certain smooth cost penalizing extreme control inputs. In the current paper, we consider an online setting where the state costs may change arbitrarily between consecutive rounds, and the learner only observes the costs at the end of each respective round. We are interested in constructing algorithms for the learner that guarantee small regret against the best stationary control policy chosen in full knowledge of the cost sequence. Our main result is showing that the smoothness of the control cost enables the simple algorithm of following the leader to achieve a regret of order $\log^2 T$ after $T$ rounds, vastly improving on the best known regret bound of order $T^{3/4}$ for this setting. △ Less

Submitted 6 June, 2017; v1 submitted 21 February, 2017; originally announced February 2017.

Action selection in growing state spaces: Control of Network Structure Growth

Authors: Dominik Thalmeier, Vicenç Gómez, Hilbert J. Kappen

Abstract: The dynamical processes taking place on a network depend on its topology. Influencing the growth process of a network therefore has important implications on such dynamical processes. We formulate the problem of influencing the growth of a network as a stochastic optimal control problem in which a structural cost function penalizes undesired topologies. We approximate this control problem with a r… ▽ More The dynamical processes taking place on a network depend on its topology. Influencing the growth process of a network therefore has important implications on such dynamical processes. We formulate the problem of influencing the growth of a network as a stochastic optimal control problem in which a structural cost function penalizes undesired topologies. We approximate this control problem with a restricted class of control problems that can be solved using probabilistic inference methods. To deal with the increasing problem dimensionality, we introduce an adaptive importance sampling method for approximating the optimal control. We illustrate this methodology in the context of formation of information cascades, considering the task of influencing the structure of a growing conversation thread, as in Internet forums. Using a realistic model of growing trees, we show that our approach can yield conversation threads with better structural properties than the ones observed without control. △ Less

Submitted 27 December, 2016; v1 submitted 23 June, 2016; originally announced June 2016.

Comments: 23 pages, 7 figures

Journal ref: Journal of Physics A: Mathematical and Theoretical, Volume 50, Number 3, 034006, 2017

Hierarchical Linearly-Solvable Markov Decision Problems

Authors: Anders Jonsson, Vicenç Gómez

Abstract: We present a hierarchical reinforcement learning framework that formulates each task in the hierarchy as a special type of Markov decision process for which the Bellman equation is linear and has analytical solution. Problems of this type, called linearly-solvable MDPs (LMDPs) have interesting properties that can be exploited in a hierarchical setting, such as efficient learning of the optimal val… ▽ More We present a hierarchical reinforcement learning framework that formulates each task in the hierarchy as a special type of Markov decision process for which the Bellman equation is linear and has analytical solution. Problems of this type, called linearly-solvable MDPs (LMDPs) have interesting properties that can be exploited in a hierarchical setting, such as efficient learning of the optimal value function or task compositionality. The proposed hierarchical approach can also be seen as a novel alternative to solving LMDPs with large state spaces. We derive a hierarchical version of the so-called Z-learning algorithm that learns different tasks simultaneously and show empirically that it significantly outperforms the state-of-the-art learning methods in two classical hierarchical reinforcement learning domains: the taxi domain and an autonomous guided vehicle task. △ Less

Submitted 10 March, 2016; originally announced March 2016.

Comments: 11 pages, 6 figures, 26th International Conference on Automated Planning and Scheduling

An Asymptotically-Optimal Sampling-Based Algorithm for Bi-directional Motion Planning

Authors: Joseph A. Starek, Javier V. Gomez, Edward Schmerling, Lucas Janson, Luis Moreno, Marco Pavone

Abstract: Bi-directional search is a widely used strategy to increase the success and convergence rates of sampling-based motion planning algorithms. Yet, few results are available that merge both bi-directional search and asymptotic optimality into existing optimal planners, such as PRM*, RRT*, and FMT*. The objective of this paper is to fill this gap. Specifically, this paper presents a bi-directional, sa… ▽ More Bi-directional search is a widely used strategy to increase the success and convergence rates of sampling-based motion planning algorithms. Yet, few results are available that merge both bi-directional search and asymptotic optimality into existing optimal planners, such as PRM*, RRT*, and FMT*. The objective of this paper is to fill this gap. Specifically, this paper presents a bi-directional, sampling-based, asymptotically-optimal algorithm named Bi-directional FMT* (BFMT*) that extends the Fast Marching Tree (FMT*) algorithm to bi-directional search while preserving its key properties, chiefly lazy search and asymptotic optimality through convergence in probability. BFMT* performs a two-source, lazy dynamic programming recursion over a set of randomly-drawn samples, correspondingly generating two search trees: one in cost-to-come space from the initial configuration and another in cost-to-go space from the goal configuration. Numerical experiments illustrate the advantages of BFMT* over its unidirectional counterpart, as well as a number of other state-of-the-art planners. △ Less

Submitted 27 July, 2015; originally announced July 2015.

Comments: Accepted to the 2015 IEEE Intelligent Robotics and Systems Conference in Hamburg, Germany. This submission represents the long version of the conference manuscript, with additional proof details (Section IV) regarding the asymptotic optimality of the BFMT* algorithm

Fast Methods for Eikonal Equations: an Experimental Survey

Authors: Javier V. Gomez, David Alvarez, Santiago Garrido, Luis Moreno

Abstract: The Fast Marching Method is a very popular algorithm to compute times-of-arrival maps (distances map measured in time units). Since their proposal in 1995, it has been applied to many different applications such as robotics, medical computer vision, fluid simulation, etc. Many alternatives have been proposed with two main objectives: to reduce its computational time and to improve its accuracy. In… ▽ More The Fast Marching Method is a very popular algorithm to compute times-of-arrival maps (distances map measured in time units). Since their proposal in 1995, it has been applied to many different applications such as robotics, medical computer vision, fluid simulation, etc. Many alternatives have been proposed with two main objectives: to reduce its computational time and to improve its accuracy. In this paper, we collect the main approaches which improve the computational time of the standard Fast Marching Method, focusing on single-threaded methods and isotropic environments. 9 different methods are studied under a common mathematical framework and experimentally in representative environments: Fast Marching Method with binary heap, Fast Marching Method with Fibonacci Heap, Simplified Fast Marching Method, Untidy Fast Marching Method, Fast Iterative Method, Group Marching Method, Fast Sweeping Method, Lock Sweeping Method and Double Dynamic Queue Method. △ Less

Submitted 11 June, 2015; originally announced June 2015.

Real-Time Stochastic Optimal Control for Multi-agent Quadrotor Systems

Authors: Vicenç Gómez, Sep Thijssen, Andrew Symington, Stephen Hailes, Hilbert J. Kappen

Abstract: This paper presents a novel method for controlling teams of unmanned aerial vehicles using Stochastic Optimal Control (SOC) theory. The approach consists of a centralized high-level planner that computes optimal state trajectories as velocity sequences, and a platform-specific low-level controller which ensures that these velocity sequences are met. The planning task is expressed as a centralized… ▽ More This paper presents a novel method for controlling teams of unmanned aerial vehicles using Stochastic Optimal Control (SOC) theory. The approach consists of a centralized high-level planner that computes optimal state trajectories as velocity sequences, and a platform-specific low-level controller which ensures that these velocity sequences are met. The planning task is expressed as a centralized path-integral control problem, for which optimal control computation corresponds to a probabilistic inference problem that can be solved by efficient sampling methods. Through simulation we show that our SOC approach (a) has significant benefits compared to deterministic control and other SOC methods in multimodal problems with noise-dependent optimal solutions, (b) is capable of controlling a large number of platforms in real-time, and (c) yields collective emergent behaviour in the form of flight formations. Finally, we show that our approach works for real platforms, by controlling a team of three quadrotors in outdoor conditions. △ Less

Submitted 12 May, 2020; v1 submitted 16 February, 2015; originally announced February 2015.

Comments: 17 pages, 8 figures, 26th International Conference on Automated Planning and Scheduling

Approximate inference on planar graphs using Loop Calculus and Belief Propagation

Authors: Vicenc Gomez, Hilbert Kappen, Michael Chertkov

Abstract: We introduce novel results for approximate inference on planar graphical models using the loop calculus framework. The loop calculus (Chertkov and Chernyak, 2006b) allows to express the exact partition function Z of a graphical model as a finite sum of terms that can be evaluated once the belief propagation (BP) solution is known. In general, full summation over all correction terms is intractable… ▽ More We introduce novel results for approximate inference on planar graphical models using the loop calculus framework. The loop calculus (Chertkov and Chernyak, 2006b) allows to express the exact partition function Z of a graphical model as a finite sum of terms that can be evaluated once the belief propagation (BP) solution is known. In general, full summation over all correction terms is intractable. We develop an algorithm for the approach presented in Chertkov et al. (2008) which represents an efficient truncation scheme on planar graphs and a new representation of the series in terms of Pfaffians of matrices. We analyze in detail both the loop series and the Pfaffian series for models with binary variables and pairwise interactions, and show that the first term of the Pfaffian series can provide very accurate approximations. The algorithm outperforms previous truncation schemes of the loop series and is competitive with other state-of-the-art methods for approximate inference. △ Less

Submitted 9 August, 2014; originally announced August 2014.

Comments: Appears in Proceedings of the Twenty-Fifth Conference on Uncertainty in Artificial Intelligence (UAI2009)

Report number: UAI-P-2009-PG-195-202

Latent Kullback Leibler Control for Continuous-State Systems using Probabilistic Graphical Models

Authors: Takamitsu Matsubara, Vicenç Gómez, Hilbert J. Kappen

Abstract: Kullback Leibler (KL) control problems allow for efficient computation of optimal control by solving a principal eigenvector problem. However, direct applicability of such framework to continuous state-action systems is limited. In this paper, we propose to embed a KL control problem in a probabilistic graphical model where observed variables correspond to the continuous (possibly high-dimensional… ▽ More Kullback Leibler (KL) control problems allow for efficient computation of optimal control by solving a principal eigenvector problem. However, direct applicability of such framework to continuous state-action systems is limited. In this paper, we propose to embed a KL control problem in a probabilistic graphical model where observed variables correspond to the continuous (possibly high-dimensional) state of the system and latent variables correspond to a discrete (low-dimensional) representation of the state amenable for KL control computation. We present two examples of this approach. The first one uses standard hidden Markov models (HMMs) and computes exact optimal control, but is only applicable to low-dimensional systems. The second one uses factorial HMMs, it is scalable to higher dimensional problems, but control computation is approximate. We illustrate both examples in several robot motor control tasks. △ Less

Submitted 27 August, 2014; v1 submitted 4 June, 2014; originally announced June 2014.

Comments: 9 pages, 5 figures, accepted in Uncertainty in Artificial Intelligence (UAI '14)

ACM Class: I.2.8; I.2.9; G.3

Learning Price-Elasticity of Smart Consumers in Power Distribution Systems

Authors: Vicenç Gómez, Michael Chertkov, Scott Backhaus, Hilbert J. Kappen

Abstract: Demand Response is an emerging technology which will transform the power grid of tomorrow. It is revolutionary, not only because it will enable peak load shaving and will add resources to manage large distribution systems, but mainly because it will tap into an almost unexplored and extremely powerful pool of resources comprised of many small individual consumers on distribution grids. However, to… ▽ More Demand Response is an emerging technology which will transform the power grid of tomorrow. It is revolutionary, not only because it will enable peak load shaving and will add resources to manage large distribution systems, but mainly because it will tap into an almost unexplored and extremely powerful pool of resources comprised of many small individual consumers on distribution grids. However, to utilize these resources effectively, the methods used to engage these resources must yield accurate and reliable control. A diversity of methods have been proposed to engage these new resources. As opposed to direct load control, many methods rely on consumers and/or loads responding to exogenous signals, typically in the form of energy pricing, originating from the utility or system operator. Here, we propose an open loop communication-lite method for estimating the price elasticity of many customers comprising a distribution system. We utilize a sparse linear regression method that relies on operator-controlled, inhomogeneous minor price variations, which will be fair to all the consumers. Our numerical experiments show that reliable estimation of individual and thus aggregated instantaneous elasticities is possible. We describe the limits of the reliable reconstruction as functions of the three key parameters of the system: (i) ratio of the number of communication slots (time units) per number of engaged consumers; (ii) level of sparsity (in consumer response); and (iii) signal-to-noise ratio. △ Less

Submitted 25 September, 2012; originally announced September 2012.

Comments: 6 pages, 5 figures, IEEE SmartGridComm 2012

ACM Class: C.2.1; G.3

A likelihood-based framework for the analysis of discussion threads

Authors: Vicenç Gómez, Hilbert J. Kappen, Nelly Litvak, Andreas Kaltenbrunner

Abstract: Online discussion threads are conversational cascades in the form of posted messages that can be generally found in social systems that comprise many-to-many interaction such as blogs, news aggregators or bulletin board systems. We propose a framework based on generative models of growing trees to analyse the structure and evolution of discussion threads. We consider the growth of a discussion to… ▽ More Online discussion threads are conversational cascades in the form of posted messages that can be generally found in social systems that comprise many-to-many interaction such as blogs, news aggregators or bulletin board systems. We propose a framework based on generative models of growing trees to analyse the structure and evolution of discussion threads. We consider the growth of a discussion to be determined by an interplay between popularity, novelty and a trend (or bias) to reply to the thread originator. The relevance of these features is estimated using a full likelihood approach and allows to characterize the habits and communication patterns of a given platform and/or community. △ Less

Submitted 3 March, 2012; originally announced March 2012.

Comments: 31 pages, 12 figures, journal

ACM Class: G.3; H.5.4

The Variational Garrote

Authors: Hilbert J. Kappen, Vicenç Gómez

Abstract: In this paper, we present a new variational method for sparse regression using $L_0$ regularization. The variational parameters appear in the approximate model in a way that is similar to Breiman's Garrote model. We refer to this method as the variational Garrote (VG). We show that the combination of the variational approximation and $L_0$ regularization has the effect of making the problem effect… ▽ More In this paper, we present a new variational method for sparse regression using $L_0$ regularization. The variational parameters appear in the approximate model in a way that is similar to Breiman's Garrote model. We refer to this method as the variational Garrote (VG). We show that the combination of the variational approximation and $L_0$ regularization has the effect of making the problem effectively of maximal rank even when the number of samples is small compared to the number of variables. The VG is compared numerically with the Lasso method, ridge regression and the recently introduced paired mean field method (PMF) (M. Titsias & M. Lázaro-Gredilla., NIPS 2012). Numerical results show that the VG and PMF yield more accurate predictions and more accurately reconstruct the true model than the other methods. It is shown that the VG finds correct solutions when the Lasso solution is inconsistent due to large input correlations. Globally, VG is significantly faster than PMF and tends to perform better as the problems become denser and in problems with strongly correlated inputs. The naive implementation of the VG scales cubic with the number of features. By introducing Lagrange multipliers we obtain a dual formulation of the problem that scales cubic in the number of samples, but close to linear in the number of features. △ Less

Submitted 12 November, 2012; v1 submitted 2 September, 2011; originally announced September 2011.

Comments: 26 pages, 11 figures

Modeling the structure and evolution of discussion cascades

Authors: Vicenç Gómez, Hilbert J. Kappen, Andreas Kaltenbrunner

Abstract: We analyze the structure and evolution of discussion cascades in four popular websites: Slashdot, Barrapunto, Meneame and Wikipedia. Despite the big heterogeneities between these sites, a preferential attachment (PA) model with bias to the root can capture the temporal evolution of the observed trees and many of their statistical properties, namely, probability distributions of the branching facto… ▽ More We analyze the structure and evolution of discussion cascades in four popular websites: Slashdot, Barrapunto, Meneame and Wikipedia. Despite the big heterogeneities between these sites, a preferential attachment (PA) model with bias to the root can capture the temporal evolution of the observed trees and many of their statistical properties, namely, probability distributions of the branching factors (degrees), subtree sizes and certain correlations. The parameters of the model are learned efficiently using a novel maximum likelihood estimation scheme for PA and provide a figurative interpretation about the communication habits and the resulting discussion cascades on the four different websites. △ Less

Submitted 15 April, 2011; v1 submitted 2 November, 2010; originally announced November 2010.

Comments: 10 pages, 11 figures

ACM Class: J.4; G.2.2

Journal ref: 22nd ACM conference on hypertext and hypermedia (HT 2011)

Dynamic Policy Programming

Authors: Mohammad Gheshlaghi Azar, Vicenc Gomez, Hilbert J. Kappen

Abstract: In this paper, we propose a novel policy iteration method, called dynamic policy programming (DPP), to estimate the optimal policy in the infinite-horizon Markov decision processes. We prove the finite-iteration and asymptotic l\infty-norm performance-loss bounds for DPP in the presence of approximation/estimation error. The bounds are expressed in terms of the l\infty-norm of the average accumula… ▽ More In this paper, we propose a novel policy iteration method, called dynamic policy programming (DPP), to estimate the optimal policy in the infinite-horizon Markov decision processes. We prove the finite-iteration and asymptotic l\infty-norm performance-loss bounds for DPP in the presence of approximation/estimation error. The bounds are expressed in terms of the l\infty-norm of the average accumulated error as opposed to the l\infty-norm of the error in the case of the standard approximate value iteration (AVI) and the approximate policy iteration (API). This suggests that DPP can achieve a better performance than AVI and API since it averages out the simulation noise caused by Monte-Carlo sampling throughout the learning process. We examine this theoretical results numerically by com- paring the performance of the approximate variants of DPP with existing reinforcement learning (RL) methods on different problem domains. Our results show that, in all cases, DPP-based algorithms outperform other RL methods by a wide margin. △ Less

Submitted 6 September, 2011; v1 submitted 12 April, 2010; originally announced April 2010.

Comments: Submitted to Journal of Machine Learning Research

Approximate inference on planar graphs using Loop Calculus and Belief Propagation

Authors: V. Gómez, H. J. Kappen, M. Chertkov

Abstract: We introduce novel results for approximate inference on planar graphical models using the loop calculus framework. The loop calculus (Chertkov and Chernyak, 2006) allows to express the exact partition function of a graphical model as a finite sum of terms that can be evaluated once the belief propagation (BP) solution is known. In general, full summation over all correction terms is intractable.… ▽ More We introduce novel results for approximate inference on planar graphical models using the loop calculus framework. The loop calculus (Chertkov and Chernyak, 2006) allows to express the exact partition function of a graphical model as a finite sum of terms that can be evaluated once the belief propagation (BP) solution is known. In general, full summation over all correction terms is intractable. We develop an algorithm for the approach presented in (Certkov et al., 2008) which represents an efficient truncation scheme on planar graphs and a new representation of the series in terms of Pfaffians of matrices. We analyze the performance of the algorithm for the partition function approximation for models with binary variables and pairwise interactions on grids and other planar graphs. We study in detail both the loop series and the equivalent Pfaffian series and show that the first term of the Pfaffian series for the general, intractable planar model, can provide very accurate approximations. The algorithm outperforms previous truncation schemes of the loop series and is competitive with other state-of-the-art methods for approximate inference. △ Less

Submitted 25 May, 2009; v1 submitted 7 January, 2009; originally announced January 2009.

Comments: 23 pages, 10 figures. Submitted to Journal of Machine Learning Research. Proceedings version accepted for UAI 2009

Homogeneous temporal activity patterns in a large online communication space

Authors: Andreas Kaltenbrunner, Vicenç Gómez, Ayman Moghnieh, Rodrigo Meza, Josep Blat, Vicente López

Abstract: The many-to-many social communication activity on the popular technology-news website Slashdot has been studied. We have concentrated on the dynamics of message production without considering semantic relations and have found regular temporal patterns in the reaction time of the community to a news-post as well as in single user behavior. The statistics of these activities follow log-normal dist… ▽ More The many-to-many social communication activity on the popular technology-news website Slashdot has been studied. We have concentrated on the dynamics of message production without considering semantic relations and have found regular temporal patterns in the reaction time of the community to a news-post as well as in single user behavior. The statistics of these activities follow log-normal distributions. Daily and weekly oscillatory cycles, which cause slight variations of this simple behavior, are identified. A superposition of two log-normal distributions can account for these variations. The findings are remarkable since the distribution of the number of comments per users, which is also analyzed, indicates a great amount of heterogeneity in the community. The reader may find surprising that only a few parameters allow a detailed description, or even prediction, of social many-to-many information exchange in this kind of popular public spaces. △ Less

Submitted 11 August, 2007; originally announced August 2007.

Comments: 17 pages, 10 figures, to be published in International Journal on WWW/Internet

ACM Class: C.2.3

Truncating the loop series expansion for Belief Propagation

Authors: Vicenc Gomez, J. M. Mooij, H. J. Kappen

Abstract: Recently, M. Chertkov and V.Y. Chernyak derived an exact expression for the partition sum (normalization constant) corresponding to a graphical model, which is an expansion around the Belief Propagation solution. By adding correction terms to the BP free energy, one for each "generalized loop" in the factor graph, the exact partition sum is obtained. However, the usually enormous number of gener… ▽ More Recently, M. Chertkov and V.Y. Chernyak derived an exact expression for the partition sum (normalization constant) corresponding to a graphical model, which is an expansion around the Belief Propagation solution. By adding correction terms to the BP free energy, one for each "generalized loop" in the factor graph, the exact partition sum is obtained. However, the usually enormous number of generalized loops generally prohibits summation over all correction terms. In this article we introduce Truncated Loop Series BP (TLSBP), a particular way of truncating the loop series of M. Chertkov and V.Y. Chernyak by considering generalized loops as compositions of simple loops. We analyze the performance of TLSBP in different scenarios, including the Ising model, regular random graphs and on Promedas, a large probabilistic medical diagnostic system. We show that TLSBP often improves upon the accuracy of the BP solution, at the expense of increased computation time. We also show that the performance of TLSBP strongly depends on the degree of interaction between the variables. For weak interactions, truncating the series leads to significant improvements, whereas for strong interactions it can be ineffective, even if a high number of terms is considered. △ Less

Submitted 25 July, 2007; v1 submitted 21 December, 2006; originally announced December 2006.

Comments: 31 pages, 12 figures, submitted to Journal of Machine Learning Research

Journal ref: The Journal of Machine Learning Research, 8(Sep):1987--2016, 2007