-
VHAKG: A Multi-modal Knowledge Graph Based on Synchronized Multi-view Videos of Daily Activities
Authors:
Shusaku Egami,
Takahiro Ugai,
Swe Nwe Nwe Htun,
Ken Fukuda
Abstract:
Multi-modal knowledge graphs (MMKGs), which ground various non-symbolic data (e.g., images and videos) into symbols, have attracted attention as resources enabling knowledge processing and machine learning across modalities. However, the construction of MMKGs for videos consisting of multiple events, such as daily activities, is still in the early stages. In this paper, we construct an MMKG based…
▽ More
Multi-modal knowledge graphs (MMKGs), which ground various non-symbolic data (e.g., images and videos) into symbols, have attracted attention as resources enabling knowledge processing and machine learning across modalities. However, the construction of MMKGs for videos consisting of multiple events, such as daily activities, is still in the early stages. In this paper, we construct an MMKG based on synchronized multi-view simulated videos of daily activities. Besides representing the content of daily life videos as event-centric knowledge, our MMKG also includes frame-by-frame fine-grained changes, such as bounding boxes within video frames. In addition, we provide support tools for querying our MMKG. As an application example, we demonstrate that our MMKG facilitates benchmarking vision-language models by providing the necessary vision-language datasets for a tailored task.
△ Less
Submitted 27 August, 2024; v1 submitted 27 August, 2024;
originally announced August 2024.
-
Multimodal Datasets and Benchmarks for Reasoning about Dynamic Spatio-Temporality in Everyday Environments
Authors:
Takanori Ugai,
Kensho Hara,
Shusaku Egami,
Ken Fukuda
Abstract:
We used a 3D simulator to create artificial video data with standardized annotations, aiming to aid in the development of Embodied AI. Our question answering (QA) dataset measures the extent to which a robot can understand human behavior and the environment in a home setting. Preliminary experiments suggest our dataset is useful in measuring AI's comprehension of daily life. \end{abstract}
We used a 3D simulator to create artificial video data with standardized annotations, aiming to aid in the development of Embodied AI. Our question answering (QA) dataset measures the extent to which a robot can understand human behavior and the environment in a home setting. Preliminary experiments suggest our dataset is useful in measuring AI's comprehension of daily life. \end{abstract}
△ Less
Submitted 21 August, 2024;
originally announced August 2024.
-
GothX: a generator of customizable, legitimate and malicious IoT network traffic
Authors:
Manuel Poisson,
Rodrigo Carnier,
Kensuke Fukuda
Abstract:
In recent years, machine learning-based anomaly detection (AD) has become an important measure against security threats from Internet of Things (IoT) networks. Machine learning (ML) models for network traffic AD require datasets to be trained, evaluated and compared. Due to the necessity of realistic and up-to-date representation of IoT security threats, new datasets need to be constantly generate…
▽ More
In recent years, machine learning-based anomaly detection (AD) has become an important measure against security threats from Internet of Things (IoT) networks. Machine learning (ML) models for network traffic AD require datasets to be trained, evaluated and compared. Due to the necessity of realistic and up-to-date representation of IoT security threats, new datasets need to be constantly generated to train relevant AD models. Since most traffic generation setups are developed considering only the author's use, replication of traffic generation becomes an additional challenge to the creation and maintenance of useful datasets. In this work, we propose GothX, a flexible traffic generator to create both legitimate and malicious traffic for IoT datasets. As a fork of Gotham Testbed, GothX is developed with five requirements: 1)easy configuration of network topology, 2) customization of traffic parameters, 3) automatic execution of legitimate and attack scenarios, 4) IoT network heterogeneity (the current iteration supports MQTT, Kafka and SINETStream services), and 5) automatic labeling of generated datasets. GothX is validated by two use cases: a) re-generation and enrichment of traffic from the IoT dataset MQTTset,and b) automatic execution of a new realistic scenario including the exploitation of a CVE specific to the Kafka-MQTT network topology and leading to a DDoS attack. We also contribute with two datasets containing mixed traffic, one made from the enriched MQTTset traffic and another from the attack scenario. We evaluated the scalability of GothX (450 IoT sensors in a single machine), the replication of the use cases and the validity of the generated datasets, confirming the ability of GothX to improve the current state-of-the-art of network traffic generation.
△ Less
Submitted 24 July, 2024; v1 submitted 10 July, 2024;
originally announced July 2024.
-
Deep Bayesian Filter for Bayes-faithful Data Assimilation
Authors:
Yuta Tarumi,
Keisuke Fukuda,
Shin-ichi Maeda
Abstract:
State estimation for nonlinear state space models is a challenging task. Existing assimilation methodologies predominantly assume Gaussian posteriors on physical space, where true posteriors become inevitably non-Gaussian. We propose Deep Bayesian Filtering (DBF) for data assimilation on nonlinear state space models (SSMs). DBF constructs new latent variables $h_t$ on a new latent (``fancy'') spac…
▽ More
State estimation for nonlinear state space models is a challenging task. Existing assimilation methodologies predominantly assume Gaussian posteriors on physical space, where true posteriors become inevitably non-Gaussian. We propose Deep Bayesian Filtering (DBF) for data assimilation on nonlinear state space models (SSMs). DBF constructs new latent variables $h_t$ on a new latent (``fancy'') space and assimilates observations $o_t$. By (i) constraining the state transition on fancy space to be linear and (ii) learning a Gaussian inverse observation operator $q(h_t|o_t)$, posteriors always remain Gaussian for DBF. Quite distinctively, the structured design of posteriors provides an analytic formula for the recursive computation of posteriors without accumulating Monte-Carlo sampling errors over time steps. DBF seeks the Gaussian inverse observation operators $q(h_t|o_t)$ and other latent SSM parameters (e.g., dynamics matrix) by maximizing the evidence lower bound. Experiments show that DBF outperforms model-based approaches and latent assimilation methods in various tasks and conditions.
△ Less
Submitted 28 May, 2024;
originally announced May 2024.
-
GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warping
Authors:
Junyoung Seo,
Kazumi Fukuda,
Takashi Shibuya,
Takuya Narihira,
Naoki Murata,
Shoukang Hu,
Chieh-Hsin Lai,
Seungryong Kim,
Yuki Mitsufuji
Abstract:
Generating novel views from a single image remains a challenging task due to the complexity of 3D scenes and the limited diversity in the existing multi-view datasets to train a model on. Recent research combining large-scale text-to-image (T2I) models with monocular depth estimation (MDE) has shown promise in handling in-the-wild images. In these methods, an input view is geometrically warped to…
▽ More
Generating novel views from a single image remains a challenging task due to the complexity of 3D scenes and the limited diversity in the existing multi-view datasets to train a model on. Recent research combining large-scale text-to-image (T2I) models with monocular depth estimation (MDE) has shown promise in handling in-the-wild images. In these methods, an input view is geometrically warped to novel views with estimated depth maps, then the warped image is inpainted by T2I models. However, they struggle with noisy depth maps and loss of semantic details when warping an input view to novel viewpoints. In this paper, we propose a novel approach for single-shot novel view synthesis, a semantic-preserving generative warping framework that enables T2I generative models to learn where to warp and where to generate, through augmenting cross-view attention with self-attention. Our approach addresses the limitations of existing methods by conditioning the generative model on source view images and incorporating geometric warping signals. Qualitative and quantitative evaluations demonstrate that our model outperforms existing methods in both in-domain and out-of-domain scenarios. Project page is available at https://GenWarp-NVS.github.io/.
△ Less
Submitted 27 May, 2024;
originally announced May 2024.
-
Synthetic Multimodal Dataset for Empowering Safety and Well-being in Home Environments
Authors:
Takanori Ugai,
Shusaku Egami,
Swe Nwe Nwe Htun,
Kouji Kozaki,
Takahiro Kawamura,
Ken Fukuda
Abstract:
This paper presents a synthetic multimodal dataset of daily activities that fuses video data from a 3D virtual space simulator with knowledge graphs depicting the spatiotemporal context of the activities. The dataset is developed for the Knowledge Graph Reasoning Challenge for Social Issues (KGRC4SI), which focuses on identifying and addressing hazardous situations in the home environment. The dat…
▽ More
This paper presents a synthetic multimodal dataset of daily activities that fuses video data from a 3D virtual space simulator with knowledge graphs depicting the spatiotemporal context of the activities. The dataset is developed for the Knowledge Graph Reasoning Challenge for Social Issues (KGRC4SI), which focuses on identifying and addressing hazardous situations in the home environment. The dataset is available to the public as a valuable resource for researchers and practitioners developing innovative solutions recognizing human behaviors to enhance safety and well-being in
△ Less
Submitted 26 January, 2024;
originally announced January 2024.
-
RDF-star2Vec: RDF-star Graph Embeddings for Data Mining
Authors:
Shusaku Egami,
Takanori Ugai,
Masateru Oota,
Kyoumoto Matsushita,
Takahiro Kawamura,
Kouji Kozaki,
Ken Fukuda
Abstract:
Knowledge Graphs (KGs) such as Resource Description Framework (RDF) data represent relationships between various entities through the structure of triples (<subject, predicate, object>). Knowledge graph embedding (KGE) is crucial in machine learning applications, specifically in node classification and link prediction tasks. KGE remains a vital research topic within the semantic web community. RDF…
▽ More
Knowledge Graphs (KGs) such as Resource Description Framework (RDF) data represent relationships between various entities through the structure of triples (<subject, predicate, object>). Knowledge graph embedding (KGE) is crucial in machine learning applications, specifically in node classification and link prediction tasks. KGE remains a vital research topic within the semantic web community. RDF-star introduces the concept of a quoted triple (QT), a specific form of triple employed either as the subject or object within another triple. Moreover, RDF-star permits a QT to act as compositional entities within another QT, thereby enabling the representation of recursive, hyper-relational KGs with nested structures. However, existing KGE models fail to adequately learn the semantics of QTs and entities, primarily because they do not account for RDF-star graphs containing multi-leveled nested QTs and QT-QT relationships. This study introduces RDF-star2Vec, a novel KGE model specifically designed for RDF-star graphs. RDF-star2Vec introduces graph walk techniques that enable probabilistic transitions between a QT and its compositional entities. Feature vectors for QTs, entities, and relations are derived from generated sequences through the structured skip-gram model. Additionally, we provide a dataset and a benchmarking framework for data mining tasks focused on complex RDF-star graphs. Evaluative experiments demonstrated that RDF-star2Vec yielded superior performance compared to recent extensions of RDF2Vec in various tasks including classification, clustering, entity relatedness, and QT similarity.
△ Less
Submitted 25 December, 2023;
originally announced December 2023.
-
Synthesizing Event-centric Knowledge Graphs of Daily Activities Using Virtual Space
Authors:
Shusaku Egami,
Takanori Ugai,
Mikiko Oono,
Koji Kitamura,
Ken Fukuda
Abstract:
Artificial intelligence (AI) is expected to be embodied in software agents, robots, and cyber-physical systems that can understand the various contextual information of daily life in the home environment to support human behavior and decision making in various situations. Scene graph and knowledge graph (KG) construction technologies have attracted much attention for knowledge-based embodied quest…
▽ More
Artificial intelligence (AI) is expected to be embodied in software agents, robots, and cyber-physical systems that can understand the various contextual information of daily life in the home environment to support human behavior and decision making in various situations. Scene graph and knowledge graph (KG) construction technologies have attracted much attention for knowledge-based embodied question answering meeting this expectation. However, collecting and managing real data on daily activities under various experimental conditions in a physical space are quite costly, and developing AI that understands the intentions and contexts is difficult. In the future, data from both virtual spaces, where conditions can be easily modified, and physical spaces, where conditions are difficult to change, are expected to be combined to analyze daily living activities. However, studies on the KG construction of daily activities using virtual space and their application have yet to progress. The potential and challenges must still be clarified to facilitate AI development for human daily life. Thus, this study proposes the VirtualHome2KG framework to generate synthetic KGs of daily life activities in virtual space. This framework augments both the synthetic video data of daily activities and the contextual semantic data corresponding to the video contents based on the proposed event-centric schema and virtual space simulation results. Therefore, context-aware data can be analyzed, and various applications that have conventionally been difficult to develop due to the insufficient availability of relevant data and semantic information can be developed. We also demonstrate herein the utility and potential of the proposed VirtualHome2KG framework through several use cases, including the analysis of daily activities by querying, embedding, and clustering, and fall risk detection among ...
△ Less
Submitted 30 July, 2023;
originally announced July 2023.
-
ASTrack: Automatic Detection and Removal of Web Tracking Code with Minimal Functionality Loss
Authors:
Ismael Castell-Uroz,
Kensuke Fukuda,
Pere Barlet-Ros
Abstract:
Recent advances in web technologies make it more difficult than ever to detect and block web tracking systems. In this work, we propose ASTrack, a novel approach to web tracking detection and removal. ASTrack uses an abstraction of the code structure based on Abstract Syntax Trees to selectively identify web tracking functionality shared across multiple web services. This new methodology allows us…
▽ More
Recent advances in web technologies make it more difficult than ever to detect and block web tracking systems. In this work, we propose ASTrack, a novel approach to web tracking detection and removal. ASTrack uses an abstraction of the code structure based on Abstract Syntax Trees to selectively identify web tracking functionality shared across multiple web services. This new methodology allows us to: (i) effectively detect web tracking code even when using evasion techniques (e.g., obfuscation, minification, or webpackaging); and (ii) safely remove those portions of code related to tracking purposes without affecting the legitimate functionality of the website. Our evaluation with the top 10k most popular Internet domains shows that ASTrack can detect web tracking with high precision (98%), while discovering about 50k tracking code pieces and more than 3,400 new tracking URLs not previously recognized by most popular privacy-preserving tools (e.g., uBlock Origin). Moreover, ASTrack achieved a 36% reduction in functionality loss in comparison with the filter lists, one of the safest options available. Using a novel methodology that combines computer vision and manual inspection, we estimate that full functionality is preserved in more than 97% of the websites.
△ Less
Submitted 25 January, 2023;
originally announced January 2023.
-
A machine learning approach for detecting CNAME cloaking-based tracking on the Web
Authors:
Ha Dao,
Kensuke Fukuda
Abstract:
Various in-browser privacy protection techniques have been designed to protect end-users from third-party tracking. In an arms race against these counter-measures, the tracking providers developed a new technique called CNAME cloaking based tracking to avoid issues with browsers that block third-party cookies and requests. To detect this tracking technique, browser extensions require on-demand DNS…
▽ More
Various in-browser privacy protection techniques have been designed to protect end-users from third-party tracking. In an arms race against these counter-measures, the tracking providers developed a new technique called CNAME cloaking based tracking to avoid issues with browsers that block third-party cookies and requests. To detect this tracking technique, browser extensions require on-demand DNS lookup APIs. This feature is however only supported by the Firefox browser.
In this paper, we propose a supervised machine learning-based method to detect CNAME cloaking-based tracking without the on-demand DNS lookup. Our goal is to detect both sites and requests linked to CNAME cloaking-related tracking. We crawl a list of target sites and store all HTTP/HTTPS requests with their attributes. Then we label all instances automatically by looking up CNAME record of subdomain, and applying wildcard matching based on well-known tracking filter lists. After extracting features, we build a supervised classification model to distinguish site and request related to CNAME cloaking-based tracking. Our evaluation shows that the proposed approach outperforms well-known tracking filter lists: F1 scores of 0.790 for sites and 0.885 for requests. By analyzing the feature permutation importance, we demonstrate that the number of scripts and the proportion of XMLHttpRequests are discriminative for detecting sites, and the length of URL request is helpful in detecting requests. Finally, we analyze concept drift by using the 2018 dataset to train a model and obtain a reasonable performance on the 2020 dataset for detecting both sites and requests using CNAME cloaking-based tracking.
△ Less
Submitted 29 September, 2020;
originally announced September 2020.
-
Characterizing and Detecting Money Laundering Activities on the Bitcoin Network
Authors:
Yining Hu,
Suranga Seneviratne,
Kanchana Thilakarathna,
Kensuke Fukuda,
Aruna Seneviratne
Abstract:
Bitcoin is by far the most popular crypto-currency solution enabling peer-to-peer payments. Despite some studies highlighting the network does not provide full anonymity, it is still being heavily used for a wide variety of dubious financial activities such as money laundering, ponzi schemes, and ransom-ware payments. In this paper, we explore the landscape of potential money laundering activities…
▽ More
Bitcoin is by far the most popular crypto-currency solution enabling peer-to-peer payments. Despite some studies highlighting the network does not provide full anonymity, it is still being heavily used for a wide variety of dubious financial activities such as money laundering, ponzi schemes, and ransom-ware payments. In this paper, we explore the landscape of potential money laundering activities occurring across the Bitcoin network. Using data collected over three years, we create transaction graphs and provide an in-depth analysis on various graph characteristics to differentiate money laundering transactions from regular transactions. We found that the main difference between laundering and regular transactions lies in their output values and neighbourhood information. Then, we propose and evaluate a set of classifiers based on four types of graph features: immediate neighbours, curated features, deepwalk embeddings, and node2vec embeddings to classify money laundering and regular transactions. Results show that the node2vec-based classifier outperforms other classifiers in binary classification reaching an average accuracy of 92.29% and an F1-measure of 0.93 and high robustness over a 2.5-year time span. Finally, we demonstrate how effective our classifiers are in discovering unknown laundering services. The classifier performance dropped compared to binary classification, however, the prediction can be improved with simple ensemble techniques for some services.
△ Less
Submitted 27 December, 2019;
originally announced December 2019.
-
A comparison of web privacy protection techniques
Authors:
Johan Mazel,
Richard Garnier,
Kensuke Fukuda
Abstract:
A comparison of web privacy protection techniques
A comparison of web privacy protection techniques
△ Less
Submitted 19 December, 2017;
originally announced December 2017.
-
Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes
Authors:
Takuya Akiba,
Shuji Suzuki,
Keisuke Fukuda
Abstract:
We demonstrate that training ResNet-50 on ImageNet for 90 epochs can be achieved in 15 minutes with 1024 Tesla P100 GPUs. This was made possible by using a large minibatch size of 32k. To maintain accuracy with this large minibatch size, we employed several techniques such as RMSprop warm-up, batch normalization without moving averages, and a slow-start learning rate schedule. This paper also desc…
▽ More
We demonstrate that training ResNet-50 on ImageNet for 90 epochs can be achieved in 15 minutes with 1024 Tesla P100 GPUs. This was made possible by using a large minibatch size of 32k. To maintain accuracy with this large minibatch size, we employed several techniques such as RMSprop warm-up, batch normalization without moving averages, and a slow-start learning rate schedule. This paper also describes the details of the hardware and software of the system used to achieve the above performance.
△ Less
Submitted 12 November, 2017;
originally announced November 2017.
-
ChainerMN: Scalable Distributed Deep Learning Framework
Authors:
Takuya Akiba,
Keisuke Fukuda,
Shuji Suzuki
Abstract:
One of the keys for deep learning to have made a breakthrough in various fields was to utilize high computing powers centering around GPUs. Enabling the use of further computing abilities by distributed processing is essential not only to make the deep learning bigger and faster but also to tackle unsolved challenges. We present the design, implementation, and evaluation of ChainerMN, the distribu…
▽ More
One of the keys for deep learning to have made a breakthrough in various fields was to utilize high computing powers centering around GPUs. Enabling the use of further computing abilities by distributed processing is essential not only to make the deep learning bigger and faster but also to tackle unsolved challenges. We present the design, implementation, and evaluation of ChainerMN, the distributed deep learning framework we have developed. We demonstrate that ChainerMN can scale the learning process of the ResNet-50 model to the ImageNet dataset up to 128 GPUs with the parallel efficiency of 90%.
△ Less
Submitted 31 October, 2017;
originally announced October 2017.
-
On the Complexity of Polytopes in $LI(2)$
Authors:
Komei Fukuda,
May Szedlak
Abstract:
In this paper we consider polytopes given by systems of $n$ inequalities in $d$ variables, where every inequality has at most two variables with nonzero coefficient. We denote this family by $LI(2)$. We show that despite of the easy algebraic structure, polytopes in $LI(2)$ can have high complexity. We construct a polytope in $LI(2)$, whose number of vertices is almost the number of vertices of th…
▽ More
In this paper we consider polytopes given by systems of $n$ inequalities in $d$ variables, where every inequality has at most two variables with nonzero coefficient. We denote this family by $LI(2)$. We show that despite of the easy algebraic structure, polytopes in $LI(2)$ can have high complexity. We construct a polytope in $LI(2)$, whose number of vertices is almost the number of vertices of the dual cyclic polytope, the difference is a multiplicative factor of depending on $d$ and in particular independent of $n$. Moreover we show that the dual cyclic polytope can not be realized in $LI(2)$.
△ Less
Submitted 30 June, 2017;
originally announced June 2017.
-
Scaling in Internet Traffic: a 14 year and 3 day longitudinal study, with multiscale analyses and random projections
Authors:
Romain Fontugne,
Patrice Abry,
Kensuke Fukuda,
Darryl Veitch,
Kenjiro Cho,
Pierre Borgnat,
Herwig Wendt
Abstract:
In the mid-90's, it was shown that the statistics of aggregated time series from Internet traffic departed from those of traditional short range dependent models, and were instead characterized by asymptotic self-similarity. Following this seminal contribution, over the years, many studies have investigated the existence and form of scaling in Internet traffic. This contribution aims first at pres…
▽ More
In the mid-90's, it was shown that the statistics of aggregated time series from Internet traffic departed from those of traditional short range dependent models, and were instead characterized by asymptotic self-similarity. Following this seminal contribution, over the years, many studies have investigated the existence and form of scaling in Internet traffic. This contribution aims first at presenting a methodology, combining multiscale analysis (wavelet and wavelet leaders) and random projections (or sketches), permitting a precise, efficient and robust characterization of scaling which is capable of seeing through non-stationary anomalies. Second, we apply the methodology to a data set spanning an unusually long period: 14 years, from the MAWI traffic archive, thereby allowing an in-depth longitudinal analysis of the form, nature and evolutions of scaling in Internet traffic, as well as network mechanisms producing them. We also study a separate 3-day long trace to obtain complementary insight into intra-day behavior. We find that a biscaling (two ranges of independent scaling phenomena) regime is systematically observed: long-range dependence over the large scales, and multifractal-like scaling over the fine scales. We quantify the actual scaling ranges precisely, verify to high accuracy the expected relationship between the long range dependent parameter and the heavy tail parameter of the flow size distribution, and relate fine scale multifractal scaling to typical IP packet inter-arrival and to round-trip time distributions.
△ Less
Submitted 6 March, 2017;
originally announced March 2017.
-
Redundancies in Linear Systems with two Variables per Inequality
Authors:
Komei Fukuda,
May Szedlak
Abstract:
The problem of detecting and removing redundant constraints is fundamental in optimization. We focus on the case of linear programs (LPs), given by $d$ variables with $n$ inequality constraints. A constraint is called \emph{redundant}, if after its removal, the LP still has the same feasible region. The currently fastest method to detect all redundancies is due to Clarkson: it solves $n$ linear pr…
▽ More
The problem of detecting and removing redundant constraints is fundamental in optimization. We focus on the case of linear programs (LPs), given by $d$ variables with $n$ inequality constraints. A constraint is called \emph{redundant}, if after its removal, the LP still has the same feasible region. The currently fastest method to detect all redundancies is due to Clarkson: it solves $n$ linear programs, but each of them has at most $s$ constraints, where $s$ is the number of nonredundant constraints.
In this paper, we study the special case where every constraint has at most two variables with nonzero coefficients. This family, denoted by $LI(2)$, has some nice properties. Namely, as shown by Aspvall and Shiloach, given a variable $x_i$ and a value $λ$, we can test in time $O(nd)$ whether there is a feasible solution with $x_i = λ$. Hochbaum and Naor present an $O(d^2 n \log n)$ algorithm for solving the feasibility problem in $LI(2)$. Their technique makes use of the Fourier-Motzkin elimination method and the earlier mentioned result by Aspvall and Shiloach.
We present a strongly polynomial algorithm that solves redundancy detection in time $O(n d^2 s \log s)$. It uses a modification of Clarkson's algorithm, together with a revised version of Hochbaum and Naor's technique. Finally we show that dimensionality testing can be done with the same running time as solving feasibility.
△ Less
Submitted 10 October, 2016;
originally announced October 2016.
-
Combinatorial Redundancy Detection
Authors:
Komei Fukuda,
Bernd Gärtner,
May Szedlák
Abstract:
The problem of detecting and removing redundant constraints is fundamental in optimization. We focus on the case of linear programs (LPs) in dictionary form, given by $n$ equality constraints in $n+d$ variables, where the variables are constrained to be nonnegative. A variable $x_r$ is called redundant, if after removing $x_r \geq 0$ the LP still has the same feasible region. The time needed to so…
▽ More
The problem of detecting and removing redundant constraints is fundamental in optimization. We focus on the case of linear programs (LPs) in dictionary form, given by $n$ equality constraints in $n+d$ variables, where the variables are constrained to be nonnegative. A variable $x_r$ is called redundant, if after removing $x_r \geq 0$ the LP still has the same feasible region. The time needed to solve such an LP is denoted by $LP(n,d)$.
It is easy to see that solving $n+d$ LPs of the above size is sufficient to detect all redundancies. The currently fastest practical method is the one by Clarkson: it solves $n+d$ linear programs, but each of them has at most $s$ variables, where $s$ is the number of nonredundant constraints.
In the first part we show that knowing all of the finitely many dictionaries of the LP is sufficient for the purpose of redundancy detection. A dictionary is a matrix that can be thought of as an enriched encoding of a vertex in the LP. Moreover - and this is the combinatorial aspect - it is enough to know only the signs of the entries, the actual values do not matter. Concretely we show that for any variable $x_r$ one can find a dictionary, such that its sign pattern is either a redundancy or nonredundancy certificate for $x_r$.
In the second part we show that considering only the sign patterns of the dictionary, there is an output sensitive algorithm of running time $\mathcal{O}(d \cdot (n+d) \cdot s^{d-1} \cdot LP(s,d) + d \cdot s^{d} \cdot LP(n,d))$ to detect all redundancies. In the case where all constraints are in general position, the running time is $\mathcal{O}(s \cdot LP(n,d) + (n+d) \cdot LP(s,d))$, which is essentially the running time of the Clarkson method. Our algorithm extends naturally to a more general setting of arrangements of oriented topological hyperplane arrangements.
△ Less
Submitted 3 December, 2014;
originally announced December 2014.
