Showing 1–22 of 22 results for author: Rutherford, A

No Regrets: Investigating and Improving Regret Approximations for Curriculum Discovery

Authors: Alexander Rutherford, Michael Beukman, Timon Willi, Bruno Lacerda, Nick Hawes, Jakob Foerster

Abstract: What data or environments to use for training to improve downstream performance is a longstanding and very topical question in reinforcement learning. In particular, Unsupervised Environment Design (UED) methods have gained recent attention as their adaptive curricula enable agents to be robust to in- and out-of-distribution tasks. We ask to what extent these methods are themselves robust when app… ▽ More What data or environments to use for training to improve downstream performance is a longstanding and very topical question in reinforcement learning. In particular, Unsupervised Environment Design (UED) methods have gained recent attention as their adaptive curricula enable agents to be robust to in- and out-of-distribution tasks. We ask to what extent these methods are themselves robust when applied to a novel setting, closely inspired by a real-world robotics problem. Surprisingly, we find that the state-of-the-art UED methods either do not improve upon the naïve baseline of Domain Randomisation (DR), or require substantial hyperparameter tuning to do so. Our analysis shows that this is due to their underlying scoring functions failing to predict intuitive measures of ``learnability'', i.e., in finding the settings that the agent sometimes solves, but not always. Based on this, we instead directly train on levels with high learnability and find that this simple and intuitive approach outperforms UED methods and DR in several binary-outcome environments, including on our domain and the standard UED domain of Minigrid. We further introduce a new adversarial evaluation procedure for directly measuring robustness, closely mirroring the conditional value at risk (CVaR). We open-source all our code and present visualisations of final policies here: https://github.com/amacrutherford/sampling-for-learnability. △ Less

Submitted 29 August, 2024; v1 submitted 27 August, 2024; originally announced August 2024.

Learning Job Title Representation from Job Description Aggregation Network

Authors: Napat Laosaengpha, Thanit Tativannarat, Chawan Piansaddhayanon, Attapol Rutherford, Ekapol Chuangsuwanich

Abstract: Learning job title representation is a vital process for developing automatic human resource tools. To do so, existing methods primarily rely on learning the title representation through skills extracted from the job description, neglecting the rich and diverse content within. Thus, we propose an alternative framework for learning job titles through their respective job description (JD) and utiliz… ▽ More Learning job title representation is a vital process for developing automatic human resource tools. To do so, existing methods primarily rely on learning the title representation through skills extracted from the job description, neglecting the rich and diverse content within. Thus, we propose an alternative framework for learning job titles through their respective job description (JD) and utilize a Job Description Aggregator component to handle the lengthy description and bidirectional contrastive loss to account for the bidirectional relationship between the job title and its description. We evaluated the performance of our method on both in-domain and out-of-domain settings, achieving a superior performance over the skill-based approach. △ Less

Submitted 12 June, 2024; originally announced June 2024.

Comments: to be published in Findings of the Association for Computational Linguistics: ACL 2024

ThaiCoref: Thai Coreference Resolution Dataset

Authors: Pontakorn Trakuekul, Wei Qi Leong, Charin Polpanumas, Jitkapat Sawatphol, William Chandra Tjhi, Attapol T. Rutherford

Abstract: While coreference resolution is a well-established research area in Natural Language Processing (NLP), research focusing on Thai language remains limited due to the lack of large annotated corpora. In this work, we introduce ThaiCoref, a dataset for Thai coreference resolution. Our dataset comprises 777,271 tokens, 44,082 mentions and 10,429 entities across four text genres: university essays, new… ▽ More While coreference resolution is a well-established research area in Natural Language Processing (NLP), research focusing on Thai language remains limited due to the lack of large annotated corpora. In this work, we introduce ThaiCoref, a dataset for Thai coreference resolution. Our dataset comprises 777,271 tokens, 44,082 mentions and 10,429 entities across four text genres: university essays, newspapers, speeches, and Wikipedia. Our annotation scheme is built upon the OntoNotes benchmark with adjustments to address Thai-specific phenomena. Utilizing ThaiCoref, we train models employing a multilingual encoder and cross-lingual transfer techniques, achieving a best F1 score of 67.88\% on the test set. Error analysis reveals challenges posed by Thai's unique linguistic features. To benefit the NLP community, we make the dataset and the model publicly available at http://www.github.com/nlp-chula/thai-coref . △ Less

Submitted 9 June, 2024; originally announced June 2024.

Thai Universal Dependency Treebank

Authors: Panyut Sriwirote, Wei Qi Leong, Charin Polpanumas, Santhawat Thanyawong, William Chandra Tjhi, Wirote Aroonmanakun, Attapol T. Rutherford

Abstract: Automatic dependency parsing of Thai sentences has been underexplored, as evidenced by the lack of large Thai dependency treebanks with complete dependency structures and the lack of a published systematic evaluation of state-of-the-art models, especially transformer-based parsers. In this work, we address these problems by introducing Thai Universal Dependency Treebank (TUD), a new largest Thai t… ▽ More Automatic dependency parsing of Thai sentences has been underexplored, as evidenced by the lack of large Thai dependency treebanks with complete dependency structures and the lack of a published systematic evaluation of state-of-the-art models, especially transformer-based parsers. In this work, we address these problems by introducing Thai Universal Dependency Treebank (TUD), a new largest Thai treebank consisting of 3,627 trees annotated in accordance with the Universal Dependencies (UD) framework. We then benchmark dependency parsing models that incorporate pretrained transformers as encoders and train them on Thai-PUD and our TUD. The evaluation results show that most of our models can outperform other models reported in previous papers and provide insight into the optimal choices of components to include in Thai dependency parsers. The new treebank and every model's full prediction generated in our experiment are made available on a GitHub repository for further study. △ Less

Submitted 13 May, 2024; originally announced May 2024.

PhayaThaiBERT: Enhancing a Pretrained Thai Language Model with Unassimilated Loanwords

Authors: Panyut Sriwirote, Jalinee Thapiang, Vasan Timtong, Attapol T. Rutherford

Abstract: While WangchanBERTa has become the de facto standard in transformer-based Thai language modeling, it still has shortcomings in regard to the understanding of foreign words, most notably English words, which are often borrowed without orthographic assimilation into Thai in many contexts. We identify the lack of foreign vocabulary in WangchanBERTa's tokenizer as the main source of these shortcomings… ▽ More While WangchanBERTa has become the de facto standard in transformer-based Thai language modeling, it still has shortcomings in regard to the understanding of foreign words, most notably English words, which are often borrowed without orthographic assimilation into Thai in many contexts. We identify the lack of foreign vocabulary in WangchanBERTa's tokenizer as the main source of these shortcomings. We then expand WangchanBERTa's vocabulary via vocabulary transfer from XLM-R's pretrained tokenizer and pretrain a new model using the expanded tokenizer, starting from WangchanBERTa's checkpoint, on a new dataset that is larger than the one used to train WangchanBERTa. Our results show that our new pretrained model, PhayaThaiBERT, outperforms WangchanBERTa in many downstream tasks and datasets. △ Less

Submitted 28 December, 2023; v1 submitted 21 November, 2023; originally announced November 2023.

Comments: revised to fix formatting error, content unchanged

JaxMARL: Multi-Agent RL Environments in JAX

Authors: Alexander Rutherford, Benjamin Ellis, Matteo Gallici, Jonathan Cook, Andrei Lupu, Gardar Ingvarsson, Timon Willi, Akbir Khan, Christian Schroeder de Witt, Alexandra Souly, Saptarashmi Bandyopadhyay, Mikayel Samvelyan, Minqi Jiang, Robert Tjarko Lange, Shimon Whiteson, Bruno Lacerda, Nick Hawes, Tim Rocktaschel, Chris Lu, Jakob Nicolaus Foerster

Abstract: Benchmarks play an important role in the development of machine learning algorithms. For example, research in reinforcement learning (RL) has been heavily influenced by available environments and benchmarks. However, RL environments are traditionally run on the CPU, limiting their scalability with typical academic compute. Recent advancements in JAX have enabled the wider use of hardware accelerat… ▽ More Benchmarks play an important role in the development of machine learning algorithms. For example, research in reinforcement learning (RL) has been heavily influenced by available environments and benchmarks. However, RL environments are traditionally run on the CPU, limiting their scalability with typical academic compute. Recent advancements in JAX have enabled the wider use of hardware acceleration to overcome these computational hurdles, enabling massively parallel RL training pipelines and environments. This is particularly useful for multi-agent reinforcement learning (MARL) research. First of all, multiple agents must be considered at each environment step, adding computational burden, and secondly, the sample complexity is increased due to non-stationarity, decentralised partial observability, or other MARL challenges. In this paper, we present JaxMARL, the first open-source code base that combines ease-of-use with GPU enabled efficiency, and supports a large number of commonly used MARL environments as well as popular baseline algorithms. When considering wall clock time, our experiments show that per-run our JAX-based training pipeline is up to 12500x faster than existing approaches. This enables efficient and thorough evaluations, with the potential to alleviate the evaluation crisis of the field. We also introduce and benchmark SMAX, a vectorised, simplified version of the popular StarCraft Multi-Agent Challenge, which removes the need to run the StarCraft II game engine. This not only enables GPU acceleration, but also provides a more flexible MARL environment, unlocking the potential for self-play, meta-learning, and other future applications in MARL. We provide code at https://github.com/flairox/jaxmarl. △ Less

Submitted 19 December, 2023; v1 submitted 16 November, 2023; originally announced November 2023.

Longitudinal Complex Dynamics of Labour Markets Reveal Increasing Polarisation

Authors: Shahad Althobaiti, Ahmad Alabdulkareem, Judy Hanwen Shen, Iyad Rahwan, Morgan Frank, Esteban Moro, Alex Rutherford

Abstract: In this paper we conduct a longitudinal analysis of the structure of labour markets in the US over 7 decades of technological, economic and policy change. We make use of network science, natural language processing and machine learning to uncover structural changes in the labour market over time. We find a steady rate of both disappearance of jobs and a shift in the required work tasks, despite mu… ▽ More In this paper we conduct a longitudinal analysis of the structure of labour markets in the US over 7 decades of technological, economic and policy change. We make use of network science, natural language processing and machine learning to uncover structural changes in the labour market over time. We find a steady rate of both disappearance of jobs and a shift in the required work tasks, despite much technological and economic change over this time period. Machine learning is used to classify jobs as being predominantly cognitive or physical based on the textual description of the workplace tasks. We also measure increasing polarisation between these two classes of jobs, linked by the similarity of tasks, over time that could constrain workers wishing to move to different jobs. △ Less

Submitted 14 April, 2022; originally announced April 2022.

The Dynamic Resilience of Urban Labour Networks

Authors: Xiangnan Feng, Alex Rutherford

Abstract: Understanding and potentially predicting or even controlling urban labour markets represents a great challenge for workers and policy makers alike. Cities are effective engines of economic growth and prosperity and incubate complex dynamics within their labour market, and the labour markets they support demonstrate considerable diversity. This presents a challenge to policy makers who would like t… ▽ More Understanding and potentially predicting or even controlling urban labour markets represents a great challenge for workers and policy makers alike. Cities are effective engines of economic growth and prosperity and incubate complex dynamics within their labour market, and the labour markets they support demonstrate considerable diversity. This presents a challenge to policy makers who would like to optimise labour markets to benefit workers, promote economic growth and manage the impact of technological change. While much previous work has studied the economic characteristics of cities as a function of size and examined the exposure of urban economies to automation, this has often been from a static perspective. In this work we examine the structure of city job networks to uncover the diffusive properties. More specifically, we identify the occupations which are most important in promoting the diffusion of beneficial or deleterious properties. We find that these properties vary considerably with city size. △ Less

Submitted 25 February, 2022; originally announced February 2022.

Comments: 27pages, 5 figures

More Than Words: Collocation Tokenization for Latent Dirichlet Allocation Models

Authors: Jin Cheevaprawatdomrong, Alexandra Schofield, Attapol T. Rutherford

Abstract: Traditionally, Latent Dirichlet Allocation (LDA) ingests words in a collection of documents to discover their latent topics using word-document co-occurrences. However, it is unclear how to achieve the best results for languages without marked word boundaries such as Chinese and Thai. Here, we explore the use of Pearson's chi-squared test, t-statistics, and Word Pair Encoding (WPE) to produce toke… ▽ More Traditionally, Latent Dirichlet Allocation (LDA) ingests words in a collection of documents to discover their latent topics using word-document co-occurrences. However, it is unclear how to achieve the best results for languages without marked word boundaries such as Chinese and Thai. Here, we explore the use of Pearson's chi-squared test, t-statistics, and Word Pair Encoding (WPE) to produce tokens as input to the LDA model. The Chi-squared, t, and WPE tokenizers are trained on Wikipedia text to look for words that should be grouped together, such as compound nouns, proper nouns, and complex event verbs. We propose a new metric for measuring the clustering quality in settings where the vocabularies of the models differ. Based on this metric and other established metrics, we show that topics trained with merged tokens result in topic keys that are clearer, more coherent, and more effective at distinguishing topics than those unmerged models. △ Less

Submitted 24 August, 2021; originally announced August 2021.

The Network Limits of Infectious Disease Control via Occupation-Based Targeting

Authors: Demetris Avraam, Nick Obradovich, Niccoló Pescetelli, Manuel Cebrian, Alex Rutherford

Abstract: Policymakers commonly employ non-pharmaceutical interventions to manage the scale and severity of pandemics. Of non-pharmaceutical interventions, social distancing policies -- designed to reduce person-to-person pathogenic spread -- have risen to recent prominence. In particular, stay-at-home policies of the sort widely implemented around the globe in response to the COVID-19 pandemic have proven… ▽ More Policymakers commonly employ non-pharmaceutical interventions to manage the scale and severity of pandemics. Of non-pharmaceutical interventions, social distancing policies -- designed to reduce person-to-person pathogenic spread -- have risen to recent prominence. In particular, stay-at-home policies of the sort widely implemented around the globe in response to the COVID-19 pandemic have proven to be markedly effective at slowing pandemic growth. However, such blunt policy instruments, while effective, produce numerous unintended consequences, including potentially dramatic reductions in economic productivity. Here we develop methods to investigate the potential to simultaneously contain pandemic spread while also minimizing economic disruptions. We do so by incorporating both occupational and network information contained within an urban environment, information that is commonly excluded from typical pandemic control policy design. The results of our method suggest that large gains in both economic productivity and pandemic control might be had by the incorporation and consideration of simple-to-measure characteristics of the occupational contact network. However we find evidence that more sophisticated, and more privacy invasive, measures of this network do not drastically increase performance. △ Less

Submitted 20 March, 2021; originally announced March 2021.

scb-mt-en-th-2020: A Large English-Thai Parallel Corpus

Authors: Lalita Lowphansirikul, Charin Polpanumas, Attapol T. Rutherford, Sarana Nutanong

Abstract: The primary objective of our work is to build a large-scale English-Thai dataset for machine translation. We construct an English-Thai machine translation dataset with over 1 million segment pairs, curated from various sources, namely news, Wikipedia articles, SMS messages, task-based dialogs, web-crawled data and government documents. Methodology for gathering data, building parallel texts and re… ▽ More The primary objective of our work is to build a large-scale English-Thai dataset for machine translation. We construct an English-Thai machine translation dataset with over 1 million segment pairs, curated from various sources, namely news, Wikipedia articles, SMS messages, task-based dialogs, web-crawled data and government documents. Methodology for gathering data, building parallel texts and removing noisy sentence pairs are presented in a reproducible manner. We train machine translation models based on this dataset. Our models' performance are comparable to that of Google Translation API (as of May 2020) for Thai-English and outperform Google when the Open Parallel Corpus (OPUS) is included in the training data for both Thai-English and English-Thai translation. The dataset, pre-trained models, and source code to reproduce our work are available for public use. △ Less

Submitted 7 July, 2020; originally announced July 2020.

Comments: 35 pages, 4 figures

AttaCut: A Fast and Accurate Neural Thai Word Segmenter

Authors: Pattarawat Chormai, Ponrawee Prasertsom, Attapol Rutherford

Abstract: Word segmentation is a fundamental pre-processing step for Thai Natural Language Processing. The current off-the-shelf solutions are not benchmarked consistently, so it is difficult to compare their trade-offs. We conducted a speed and accuracy comparison of the popular systems on three different domains and found that the state-of-the-art deep learning system is slow and moreover does not use sub… ▽ More Word segmentation is a fundamental pre-processing step for Thai Natural Language Processing. The current off-the-shelf solutions are not benchmarked consistently, so it is difficult to compare their trade-offs. We conducted a speed and accuracy comparison of the popular systems on three different domains and found that the state-of-the-art deep learning system is slow and moreover does not use sub-word structures to guide the model. Here, we propose a fast and accurate neural Thai Word Segmenter that uses dilated CNN filters to capture the environment of each character and uses syllable embeddings as features. Our system runs at least 5.6x faster and outperforms the previous state-of-the-art system on some domains. In addition, we develop the first ML-based Thai orthographical syllable segmenter, which yields syllable embeddings to be used as features by the word segmenter. △ Less

Submitted 16 November, 2019; originally announced November 2019.

Comments: 14 pages, 7 figures, accepted as oral presentation at New in ML Workshop, NeurIPS 2019

Self-supervised Recurrent Neural Network for 4D Abdominal and In-utero MR Imaging

Authors: Tong Zhang, Laurence H. Jackson, Alena Uus, James R. Clough, Lisa Story, Mary A. Rutherford, Joseph V. Hajnal, Maria Deprez

Abstract: Accurately estimating and correcting the motion artifacts are crucial for 3D image reconstruction of the abdominal and in-utero magnetic resonance imaging (MRI). The state-of-art methods are based on slice-to-volume registration (SVR) where multiple 2D image stacks are acquired in three orthogonal orientations. In this work, we present a novel reconstruction pipeline that only needs one orientatio… ▽ More Accurately estimating and correcting the motion artifacts are crucial for 3D image reconstruction of the abdominal and in-utero magnetic resonance imaging (MRI). The state-of-art methods are based on slice-to-volume registration (SVR) where multiple 2D image stacks are acquired in three orthogonal orientations. In this work, we present a novel reconstruction pipeline that only needs one orientation of 2D MRI scans and can reconstruct the full high-resolution image without masking or registration steps. The framework consists of two main stages: the respiratory motion estimation using a self-supervised recurrent neural network, which learns the respiratory signals that are naturally embedded in the asymmetry relationship of the neighborhood slices and cluster them according to a respiratory state. Then, we train a 3D deconvolutional network for super-resolution (SR) reconstruction of the sparsely selected 2D images using integrated reconstruction and total variation loss. We evaluate the classification accuracy on 5 simulated images and compare our results with the SVR method in adult abdominal and in-utero MRI scans. The results show that the proposed pipeline can accurately estimate the respiratory state and reconstruct 4D SR volumes with better or similar performance to the 3D SVR pipeline with less than 20\% sparsely selected slices. The method has great potential to transform the 4D abdominal and in-utero MRI in clinical practice. △ Less

Submitted 28 August, 2019; originally announced August 2019.

Comments: Accepted by MICCAI 2019 workshop on Machine Learning for Medical Image Reconstruction

Mapping the Privacy-Utility Tradeoff in Mobile Phone Data for Development

Authors: Alejandro Noriega-Campero, Alex Rutherford, Oren Lederman, Yves A. de Montjoye, Alex Pentland

Abstract: Today's age of data holds high potential to enhance the way we pursue and monitor progress in the fields of development and humanitarian action. We study the relation between data utility and privacy risk in large-scale behavioral data, focusing on mobile phone metadata as paradigmatic domain. To measure utility, we survey experts about the value of mobile phone metadata at various spatial and tem… ▽ More Today's age of data holds high potential to enhance the way we pursue and monitor progress in the fields of development and humanitarian action. We study the relation between data utility and privacy risk in large-scale behavioral data, focusing on mobile phone metadata as paradigmatic domain. To measure utility, we survey experts about the value of mobile phone metadata at various spatial and temporal granularity levels. To measure privacy, we propose a formal and intuitive measure of reidentification risk$\unicode{x2014}$the information ratio$\unicode{x2014}$and compute it at each granularity level. Our results confirm the existence of a stark tradeoff between data utility and reidentifiability, where the most valuable datasets are also most prone to reidentification. When data is specified at ZIP-code and hourly levels, outside knowledge of only 7% of a person's data suffices for reidentification and retrieval of the remaining 93%. In contrast, in the least valuable dataset, specified at municipality and daily levels, reidentification requires on average outside knowledge of 51%, or 31 data points, of a person's data to retrieve the remaining 49%. Overall, our findings show that coarsening data directly erodes its value, and highlight the need for using data-coarsening, not as stand-alone mechanism, but in combination with data-sharing models that provide adjustable degrees of accountability and security. △ Less

Submitted 1 August, 2018; originally announced August 2018.

Twitter as a Source of Global Mobility Patterns for Social Good

Authors: Mark Dredze, Manuel García-Herranz, Alex Rutherford, Gideon Mann

Abstract: Data on human spatial distribution and movement is essential for understanding and analyzing social systems. However existing sources for this data are lacking in various ways; difficult to access, biased, have poor geographical or temporal resolution, or are significantly delayed. In this paper, we describe how geolocation data from Twitter can be used to estimate global mobility patterns and add… ▽ More Data on human spatial distribution and movement is essential for understanding and analyzing social systems. However existing sources for this data are lacking in various ways; difficult to access, biased, have poor geographical or temporal resolution, or are significantly delayed. In this paper, we describe how geolocation data from Twitter can be used to estimate global mobility patterns and address these shortcomings. These findings will inform how this novel data source can be harnessed to address humanitarian and development efforts. △ Less

Submitted 20 June, 2016; originally announced June 2016.

Comments: Presented at 2016 ICML Workshop on #Data4Good: Machine Learning in Social Good Applications, New York, NY

Neural Network Models for Implicit Discourse Relation Classification in English and Chinese without Surface Features

Authors: Attapol T. Rutherford, Vera Demberg, Nianwen Xue

Abstract: Inferring implicit discourse relations in natural language text is the most difficult subtask in discourse parsing. Surface features achieve good performance, but they are not readily applicable to other languages without semantic lexicons. Previous neural models require parses, surface features, or a small label set to work well. Here, we propose neural network models that are based on feedforwar… ▽ More Inferring implicit discourse relations in natural language text is the most difficult subtask in discourse parsing. Surface features achieve good performance, but they are not readily applicable to other languages without semantic lexicons. Previous neural models require parses, surface features, or a small label set to work well. Here, we propose neural network models that are based on feedforward and long-short term memory architecture without any surface features. To our surprise, our best configured feedforward architecture outperforms LSTM-based model in most cases despite thorough tuning. Under various fine-grained label sets and a cross-linguistic setting, our feedforward models perform consistently better or at least just as well as systems that require hand-crafted surface features. Our models present the first neural Chinese discourse parser in the style of Chinese Discourse Treebank, showing that our results hold cross-linguistically. △ Less

Submitted 6 June, 2016; originally announced June 2016.

DeepCut: Object Segmentation from Bounding Box Annotations using Convolutional Neural Networks

Authors: Martin Rajchl, Matthew C. H. Lee, Ozan Oktay, Konstantinos Kamnitsas, Jonathan Passerat-Palmbach, Wenjia Bai, Mellisa Damodaram, Mary A. Rutherford, Joseph V. Hajnal, Bernhard Kainz, Daniel Rueckert

Abstract: In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled with bounding box annotations. It extends the approach of the well-known GrabCut method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional… ▽ More In this paper, we propose DeepCut, a method to obtain pixelwise object segmentations given an image dataset labelled with bounding box annotations. It extends the approach of the well-known GrabCut method to include machine learning by training a neural network classifier from bounding box annotations. We formulate the problem as an energy minimisation problem over a densely-connected conditional random field and iteratively update the training targets to obtain pixelwise object segmentations. Additionally, we propose variants of the DeepCut method and compare those to a naive approach to CNN training under weak supervision. We test its applicability to solve brain and lung segmentation problems on a challenging fetal magnetic resonance dataset and obtain encouraging results in terms of accuracy. △ Less

Submitted 5 June, 2016; v1 submitted 25 May, 2016; originally announced May 2016.

The International Postal Network and Other Global Flows As Proxies for National Wellbeing

Authors: Desislava Hristova, Alex Rutherford, Jose Anson, Miguel Luengo-Oroz, Cecilia Mascolo

Abstract: The digital exhaust left by flows of physical and digital commodities provides a rich measure of the nature, strength and significance of relationships between countries in the global network. With this work, we examine how these traces and the network structure can reveal the socioeconomic profile of different countries. We take into account multiple international networks of physical and digital… ▽ More The digital exhaust left by flows of physical and digital commodities provides a rich measure of the nature, strength and significance of relationships between countries in the global network. With this work, we examine how these traces and the network structure can reveal the socioeconomic profile of different countries. We take into account multiple international networks of physical and digital flows, including the previously unexplored international postal network. By measuring the position of each country in the Trade, Postal, Migration, International Flights, IP and Digital Communications networks, we are able to build proxies for a number of crucial socioeconomic indicators such as GDP per capita and the Human Development Index ranking along with twelve other indicators used as benchmarks of national wellbeing by the United Nations and other international organisations. In this context, we have also proposed and evaluated a global connectivity degree measure applying multiplex theory across the six networks that accounts for the strength of relationships between countries. We conclude with a multiplex community analysis of the global flow networks, showing how countries with shared community membership over multiple networks have similar socioeconomic profiles. Combining multiple flow data sources into global multiplex networks can help understand the forces which drive economic activity on a global level. Such an ability to infer proxy indicators in a context of incomplete information is extremely timely in light of recent discussions on measurement of indicators relevant to the Sustainable Development Goals. △ Less

Submitted 25 January, 2016; v1 submitted 22 January, 2016; originally announced January 2016.

Estimating Food Consumption and Poverty Indices with Mobile Phone Data

Authors: Adeline Decuyper, Alex Rutherford, Amit Wadhwa, Jean-Martin Bauer, Gautier Krings, Thoralf Gutierrez, Vincent D. Blondel, Miguel A. Luengo-Oroz

Abstract: Recent studies have shown the value of mobile phone data to tackle problems related to economic development and humanitarian action. In this research, we assess the suitability of indicators derived from mobile phone data as a proxy for food security indicators. We compare the measures extracted from call detail records and airtime credit purchases to the results of a nationwide household survey c… ▽ More Recent studies have shown the value of mobile phone data to tackle problems related to economic development and humanitarian action. In this research, we assess the suitability of indicators derived from mobile phone data as a proxy for food security indicators. We compare the measures extracted from call detail records and airtime credit purchases to the results of a nationwide household survey conducted at the same time. Results show high correlations (> .8) between mobile phone data derived indicators and several relevant food security variables such as expenditure on food or vegetable consumption. This correspondence suggests that, in the future, proxies derived from mobile phone data could be used to provide valuable up-to-date operational information on food security throughout low and middle income countries. △ Less

Submitted 22 November, 2014; originally announced December 2014.

Flooding through the lens of mobile phone activity

Authors: David Pastor-Escuredo, Alfredo Morales-Guzmán, Yolanda Torres-Fernández, Jean-Martin Bauer, Amit Wadhwa, Carlos Castro-Correa, Liudmyla Romanoff, Jong Gun Lee, Alex Rutherford, Vanessa Frias-Martinez, Nuria Oliver, Enrique Frias-Martinez, Miguel Luengo-Oroz

Abstract: Natural disasters affect hundreds of millions of people worldwide every year. Emergency response efforts depend upon the availability of timely information, such as information concerning the movements of affected populations. The analysis of aggregated and anonymized Call Detail Records (CDR) captured from the mobile phone infrastructure provides new possibilities to characterize human behavior d… ▽ More Natural disasters affect hundreds of millions of people worldwide every year. Emergency response efforts depend upon the availability of timely information, such as information concerning the movements of affected populations. The analysis of aggregated and anonymized Call Detail Records (CDR) captured from the mobile phone infrastructure provides new possibilities to characterize human behavior during critical events. In this work, we investigate the viability of using CDR data combined with other sources of information to characterize the floods that occurred in Tabasco, Mexico in 2009. An impact map has been reconstructed using Landsat-7 images to identify the floods. Within this frame, the underlying communication activity signals in the CDR data have been analyzed and compared against rainfall levels extracted from data of the NASA-TRMM project. The variations in the number of active phones connected to each cell tower reveal abnormal activity patterns in the most affected locations during and after the floods that could be used as signatures of the floods - both in terms of infrastructure impact assessment and population information awareness. The representativeness of the analysis has been assessed using census data and civil protection records. While a more extensive validation is required, these early results suggest high potential in using cell tower activity information to improve early warning and emergency management mechanisms. △ Less

Submitted 24 November, 2014; originally announced November 2014.

Comments: Submitted to IEEE Global Humanitarian Technologies Conference (GHTC) 2014

Journal ref: IEEE Global Humanitarian Technology Conference (GHTC), 2014 IEEE (pp. 279-286)

Targeted Social Mobilisation in a Global Manhunt

Authors: Alex Rutherford, Manuel Cebrian, Iyad Rahwan, Sohan Dsouza, James McInerney, Victor Naroditskiy, Matteo Venanzi, Nicholas R. Jennings, J. R. deLara, Eero Wahlstedt, Steven U. Miller

Abstract: Social mobilization, the ability to mobilize large numbers of people via social networks to achieve highly distributed tasks, has received significant attention in recent times. This growing capability, facilitated by modern communication technology, is highly relevant to endeavors which require the search for individuals that posses rare information or skill, such as finding medical doctors durin… ▽ More Social mobilization, the ability to mobilize large numbers of people via social networks to achieve highly distributed tasks, has received significant attention in recent times. This growing capability, facilitated by modern communication technology, is highly relevant to endeavors which require the search for individuals that posses rare information or skill, such as finding medical doctors during disasters, or searching for missing people. An open question remains, as to whether in time-critical situations, people are able to recruit in a targeted manner, or whether they resort to so-called blind search, recruiting as many acquaintances as possible via broadcast communication. To explore this question, we examine data from our recent success in the U.S. State Department's Tag Challenge, which required locating and photographing 5 target persons in 5 different cities in the United States and Europe in less than 12 hours, based only on a single mug-shot. We find that people are able to consistently route information in a targeted fashion even under increasing time pressure. We derive an analytical model for global mobilization and use it to quantify the extent to which people were targeting others during recruitment. Our model estimates that approximately 1 in 3 messages were of targeted fashion during the most time-sensitive period of the challenge.This is a novel observation at such short temporal scales, and calls for opportunities for devising viral incentive schemes that provide distance- or time-sensitive rewards to approach the target geography more rapidly, with applications in multiple areas from emergency preparedness, to political mobilization. △ Less

Submitted 6 April, 2014; v1 submitted 18 April, 2013; originally announced April 2013.

Comments: 10 pages, 11 figures (Added Supplementary Information)

Journal ref: PLoS One (2013) 8 (9)

Good Fences: The Importance of Setting Boundaries for Peaceful Coexistence

Authors: Alex Rutherford, Dion Harmon, Justin Werfel, Shlomiya Bar-Yam, Alexander Gard-Murray, Andreas Gros, Yaneer Bar-Yam

Abstract: We consider the conditions of peace and violence among ethnic groups, testing a theory designed to predict the locations of violence and interventions that can promote peace. Characterizing the model's success in predicting peace requires examples where peace prevails despite diversity. Switzerland is recognized as a country of peace, stability and prosperity. This is surprising because of its lin… ▽ More We consider the conditions of peace and violence among ethnic groups, testing a theory designed to predict the locations of violence and interventions that can promote peace. Characterizing the model's success in predicting peace requires examples where peace prevails despite diversity. Switzerland is recognized as a country of peace, stability and prosperity. This is surprising because of its linguistic and religious diversity that in other parts of the world lead to conflict and violence. Here we analyze how peaceful stability is maintained. Our analysis shows that peace does not depend on integrated coexistence, but rather on well defined topographical and political boundaries separating groups. Mountains and lakes are an important part of the boundaries between sharply defined linguistic areas. Political canton and circle (sub-canton) boundaries often separate religious groups. Where such boundaries do not appear to be sufficient, we find that specific aspects of the population distribution either guarantee sufficient separation or sufficient mixing to inhibit intergroup violence according to the quantitative theory of conflict. In exactly one region, a porous mountain range does not adequately separate linguistic groups and violent conflict has led to the recent creation of the canton of Jura. Our analysis supports the hypothesis that violence between groups can be inhibited by physical and political boundaries. A similar analysis of the area of the former Yugoslavia shows that during widespread ethnic violence existing political boundaries did not coincide with the boundaries of distinct groups, but peace prevailed in specific areas where they did coincide. The success of peace in Switzerland may serve as a model to resolve conflict in other ethnically diverse countries and regions of the world. △ Less

Submitted 6 October, 2011; originally announced October 2011.

Comments: paper pages 1-14, 4 figures; appendices pages 15-43, 20 figures

Report number: NECSI 2011-10-01