-
Modeling Urban Transport Choices: Incorporating Sociocultural Aspects
Authors:
Kathleen Salazar-Serna,
Lorena Cadavid,
Carlos J. Franco
Abstract:
This paper introduces an agent-based simulation model aimed at understanding urban commuters mode choices and evaluating the impacts of transport policies to promote sustainable mobility. Crafted for developing countries, where utilitarian travel heavily relies on motorcycles, the model integrates sociocultural factors that influence transport behavior. Multinomial models and inferential statistic…
▽ More
This paper introduces an agent-based simulation model aimed at understanding urban commuters mode choices and evaluating the impacts of transport policies to promote sustainable mobility. Crafted for developing countries, where utilitarian travel heavily relies on motorcycles, the model integrates sociocultural factors that influence transport behavior. Multinomial models and inferential statistics applied to survey data from Cali, Colombia, inform the model, revealing significant influences of sociodemographic factors and travel attributes on mode choice. Findings highlight the importance of cost, time, safety, comfort, and personal security, with disparities across socioeconomic groups. Policy simulations demonstrate positive responses to interventions like free public transportation, increased bus frequency, and enhanced security, yet with modest shifts in mode choice. Multifaceted policy approaches are deemed more effective, addressing diverse user preferences. Outputs can be extended to cities with similar sociocultural characteristics and transport dynamics. The methodology applied in this work can be replicated for other territories.
△ Less
Submitted 30 July, 2024;
originally announced July 2024.
-
VortSDF: 3D Modeling with Centroidal Voronoi Tesselation on Signed Distance Field
Authors:
Diego Thomas,
Briac Toussaint,
Jean-Sebastien Franco,
Edmond Boyer
Abstract:
Volumetric shape representations have become ubiquitous in multi-view reconstruction tasks. They often build on regular voxel grids as discrete representations of 3D shape functions, such as SDF or radiance fields, either as the full shape model or as sampled instantiations of continuous representations, as with neural networks. Despite their proven efficiency, voxel representations come with the…
▽ More
Volumetric shape representations have become ubiquitous in multi-view reconstruction tasks. They often build on regular voxel grids as discrete representations of 3D shape functions, such as SDF or radiance fields, either as the full shape model or as sampled instantiations of continuous representations, as with neural networks. Despite their proven efficiency, voxel representations come with the precision versus complexity trade-off. This inherent limitation can significantly impact performance when moving away from simple and uncluttered scenes. In this paper we investigate an alternative discretization strategy with the Centroidal Voronoi Tesselation (CVT). CVTs allow to better partition the observation space with respect to shape occupancy and to focus the discretization around shape surfaces. To leverage this discretization strategy for multi-view reconstruction, we introduce a volumetric optimization framework that combines explicit SDF fields with a shallow color network, in order to estimate 3D shape properties over tetrahedral grids. Experimental results with Chamfer statistics validate this approach with unprecedented reconstruction quality on various scenarios such as objects, open scenes or human.
△ Less
Submitted 29 July, 2024;
originally announced July 2024.
-
A Mixture of Experts Approach to 3D Human Motion Prediction
Authors:
Edmund Shieh,
Joshua Lee Franco,
Kang Min Bae,
Tej Lalvani
Abstract:
This project addresses the challenge of human motion prediction, a critical area for applications such as au- tonomous vehicle movement detection. Previous works have emphasized the need for low inference times to provide real time performance for applications like these. Our primary objective is to critically evaluate existing model ar- chitectures, identifying their advantages and opportunities…
▽ More
This project addresses the challenge of human motion prediction, a critical area for applications such as au- tonomous vehicle movement detection. Previous works have emphasized the need for low inference times to provide real time performance for applications like these. Our primary objective is to critically evaluate existing model ar- chitectures, identifying their advantages and opportunities for improvement by replicating the state-of-the-art (SOTA) Spatio-Temporal Transformer model as best as possible given computational con- straints. These models have surpassed the limitations of RNN-based models and have demonstrated the ability to generate plausible motion sequences over both short and long term horizons through the use of spatio-temporal rep- resentations. We also propose a novel architecture to ad- dress challenges of real time inference speed by incorpo- rating a Mixture of Experts (MoE) block within the Spatial- Temporal (ST) attention layer. The particular variation that is used is Soft MoE, a fully-differentiable sparse Transformer that has shown promising ability to enable larger model capacity at lower inference cost. We make out code publicly available at https://github.com/edshieh/motionprediction
△ Less
Submitted 9 May, 2024;
originally announced May 2024.
-
PartIR: Composing SPMD Partitioning Strategies for Machine Learning
Authors:
Sami Alabed,
Daniel Belov,
Bart Chrzaszcz,
Juliana Franco,
Dominik Grewe,
Dougal Maclaurin,
James Molloy,
Tom Natan,
Tamara Norman,
Xiaoyue Pan,
Adam Paszke,
Norman A. Rink,
Michael Schaarschmidt,
Timur Sitdikov,
Agnieszka Swietlik,
Dimitrios Vytiniotis,
Joel Wee
Abstract:
Training of modern large neural networks (NN) requires a combination of parallelization strategies encompassing data, model, or optimizer sharding. When strategies increase in complexity, it becomes necessary for partitioning tools to be 1) expressive, allowing the composition of simpler strategies, and 2) predictable to estimate performance analytically. We present PartIR, our design for a NN par…
▽ More
Training of modern large neural networks (NN) requires a combination of parallelization strategies encompassing data, model, or optimizer sharding. When strategies increase in complexity, it becomes necessary for partitioning tools to be 1) expressive, allowing the composition of simpler strategies, and 2) predictable to estimate performance analytically. We present PartIR, our design for a NN partitioning system. PartIR is focused on an incremental approach to rewriting and is hardware-and-runtime agnostic. We present a simple but powerful API for composing sharding strategies and a simulator to validate them. The process is driven by high-level programmer-issued partitioning tactics, which can be both manual and automatic. Importantly, the tactics are specified separately from the model code, making them easy to change. We evaluate PartIR on several different models to demonstrate its predictability, expressibility, and ability to reach peak performance..
△ Less
Submitted 3 March, 2024; v1 submitted 20 January, 2024;
originally announced January 2024.
-
Gemini: A Family of Highly Capable Multimodal Models
Authors:
Gemini Team,
Rohan Anil,
Sebastian Borgeaud,
Jean-Baptiste Alayrac,
Jiahui Yu,
Radu Soricut,
Johan Schalkwyk,
Andrew M. Dai,
Anja Hauth,
Katie Millican,
David Silver,
Melvin Johnson,
Ioannis Antonoglou,
Julian Schrittwieser,
Amelia Glaese,
Jilin Chen,
Emily Pitler,
Timothy Lillicrap,
Angeliki Lazaridou,
Orhan Firat,
James Molloy,
Michael Isard,
Paul R. Barham,
Tom Hennigan,
Benjamin Lee
, et al. (1325 additional authors not shown)
Abstract:
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultr…
▽ More
This report introduces a new family of multimodal models, Gemini, that exhibit remarkable capabilities across image, audio, video, and text understanding. The Gemini family consists of Ultra, Pro, and Nano sizes, suitable for applications ranging from complex reasoning tasks to on-device memory-constrained use-cases. Evaluation on a broad range of benchmarks shows that our most-capable Gemini Ultra model advances the state of the art in 30 of 32 of these benchmarks - notably being the first model to achieve human-expert performance on the well-studied exam benchmark MMLU, and improving the state of the art in every one of the 20 multimodal benchmarks we examined. We believe that the new capabilities of the Gemini family in cross-modal reasoning and language understanding will enable a wide variety of use cases. We discuss our approach toward post-training and deploying Gemini models responsibly to users through services including Gemini, Gemini Advanced, Google AI Studio, and Cloud Vertex AI.
△ Less
Submitted 17 June, 2024; v1 submitted 18 December, 2023;
originally announced December 2023.
-
A Novel VAPT Algorithm: Enhancing Web Application Security Trough OWASP top 10 Optimization
Authors:
Rui Ventura,
Daniel Jose Franco,
Omar Khasro Akram
Abstract:
This research study is built upon cybersecurity audits and investigates the optimization of an Open Web Application Security Project (OWASP) Top 10 algorithm for Web Applications (WA) security audits using Vulnerability Assessment and Penetration Testing (VAPT) processes. The study places particular emphasis on enhancing the VAPT process by optimizing the OWASP algorithm. To achieve this, the rese…
▽ More
This research study is built upon cybersecurity audits and investigates the optimization of an Open Web Application Security Project (OWASP) Top 10 algorithm for Web Applications (WA) security audits using Vulnerability Assessment and Penetration Testing (VAPT) processes. The study places particular emphasis on enhancing the VAPT process by optimizing the OWASP algorithm. To achieve this, the research utilizes desk documents to gain knowledge of WA cybersecurity audits and their associated tools. It also delves into archives to explore VAPT processes and identify techniques, methods, and tools for VAPT automation. Furthermore, the research proposes a prototype optimization that streamlines the two steps of VAPT using the OWASP Top 10 algorithm through an experimental procedure. The results are obtained within a virtual environment, which employs black box testing methods as the primary means of data acquisition and analysis. In this experimental setting, the OWASP algorithm demonstrates an impressive level of precision, achieving a precision rate exceeding 90%. It effectively covers all researched vulnerabilities, thus justifying its optimization. This research contributes significantly to the enhancement of the OWASP algorithm and benefits the offensive security community. It plays a crucial role in ensuring compliance processes for professionals and analysts in the security and software development fields.
△ Less
Submitted 17 November, 2023;
originally announced November 2023.
-
SHOWMe: Benchmarking Object-agnostic Hand-Object 3D Reconstruction
Authors:
Anilkumar Swamy,
Vincent Leroy,
Philippe Weinzaepfel,
Fabien Baradel,
Salma Galaaoui,
Romain Bregier,
Matthieu Armando,
Jean-Sebastien Franco,
Gregory Rogez
Abstract:
Recent hand-object interaction datasets show limited real object variability and rely on fitting the MANO parametric model to obtain groundtruth hand shapes. To go beyond these limitations and spur further research, we introduce the SHOWMe dataset which consists of 96 videos, annotated with real and detailed hand-object 3D textured meshes. Following recent work, we consider a rigid hand-object sce…
▽ More
Recent hand-object interaction datasets show limited real object variability and rely on fitting the MANO parametric model to obtain groundtruth hand shapes. To go beyond these limitations and spur further research, we introduce the SHOWMe dataset which consists of 96 videos, annotated with real and detailed hand-object 3D textured meshes. Following recent work, we consider a rigid hand-object scenario, in which the pose of the hand with respect to the object remains constant during the whole video sequence. This assumption allows us to register sub-millimetre-precise groundtruth 3D scans to the image sequences in SHOWMe. Although simpler, this hypothesis makes sense in terms of applications where the required accuracy and level of detail is important eg., object hand-over in human-robot collaboration, object scanning, or manipulation and contact point analysis. Importantly, the rigidity of the hand-object systems allows to tackle video-based 3D reconstruction of unknown hand-held objects using a 2-stage pipeline consisting of a rigid registration step followed by a multi-view reconstruction (MVR) part. We carefully evaluate a set of non-trivial baselines for these two stages and show that it is possible to achieve promising object-agnostic 3D hand-object reconstructions employing an SfM toolbox or a hand pose estimator to recover the rigid transforms and off-the-shelf MVR algorithms. However, these methods remain sensitive to the initial camera pose estimates which might be imprecise due to lack of textures on the objects or heavy occlusions of the hands, leaving room for improvements in the reconstruction. Code and dataset are available at https://europe.naverlabs.com/research/showme
△ Less
Submitted 19 September, 2023;
originally announced September 2023.
-
Simulating the social influence in transport mode choices
Authors:
Kathleen Salazar-Serna,
Lynnette Hui Xian Ng,
Lorena Cadavid,
Carlos J. Franco,
Kathleen Carley
Abstract:
Agent-based simulations have been used in modeling transportation systems for traffic management and passenger flows. In this work, we hope to shed light on the complex factors that influence transportation mode decisions within developing countries, using Colombia as a case study. We model an ecosystem of human agents that decide at each time step on the mode of transportation they would take to…
▽ More
Agent-based simulations have been used in modeling transportation systems for traffic management and passenger flows. In this work, we hope to shed light on the complex factors that influence transportation mode decisions within developing countries, using Colombia as a case study. We model an ecosystem of human agents that decide at each time step on the mode of transportation they would take to work. Their decision is based on a combination of their personal satisfaction with the journey they had just taken, which is evaluated across a personal vector of needs, the information they crowdsource from their prevailing social network, and their personal uncertainty about the experience of trying a new transport solution. We simulate different network structures to analyze the social influence for different decision-makers. We find that in low/medium connected groups inquisitive people actively change modes cyclically over the years while imitators cluster rapidly and change less frequently.
△ Less
Submitted 1 August, 2023;
originally announced August 2023.
-
4DHumanOutfit: a multi-subject 4D dataset of human motion sequences in varying outfits exhibiting large displacements
Authors:
Matthieu Armando,
Laurence Boissieux,
Edmond Boyer,
Jean-Sebastien Franco,
Martin Humenberger,
Christophe Legras,
Vincent Leroy,
Mathieu Marsot,
Julien Pansiot,
Sergi Pujades,
Rim Rekik,
Gregory Rogez,
Anilkumar Swamy,
Stefanie Wuhrer
Abstract:
This work presents 4DHumanOutfit, a new dataset of densely sampled spatio-temporal 4D human motion data of different actors, outfits and motions. The dataset is designed to contain different actors wearing different outfits while performing different motions in each outfit. In this way, the dataset can be seen as a cube of data containing 4D motion sequences along 3 axes with identity, outfit and…
▽ More
This work presents 4DHumanOutfit, a new dataset of densely sampled spatio-temporal 4D human motion data of different actors, outfits and motions. The dataset is designed to contain different actors wearing different outfits while performing different motions in each outfit. In this way, the dataset can be seen as a cube of data containing 4D motion sequences along 3 axes with identity, outfit and motion. This rich dataset has numerous potential applications for the processing and creation of digital humans, e.g. augmented reality, avatar creation and virtual try on. 4DHumanOutfit is released for research purposes at https://kinovis.inria.fr/4dhumanoutfit/. In addition to image data and 4D reconstructions, the dataset includes reference solutions for each axis. We present independent baselines along each axis that demonstrate the value of these reference solutions for evaluation tasks.
△ Less
Submitted 12 June, 2023;
originally announced June 2023.
-
Correspondence-free online human motion retargeting
Authors:
Rim Rekik,
Mathieu Marsot,
Anne-Hélène Olivier,
Jean-Sébastien Franco,
Stefanie Wuhrer
Abstract:
We present a data-driven framework for unsupervised human motion retargeting that animates a target subject with the motion of a source subject. Our method is correspondence-free, requiring neither spatial correspondences between the source and target shapes nor temporal correspondences between different frames of the source motion. This allows to animate a target shape with arbitrary sequences of…
▽ More
We present a data-driven framework for unsupervised human motion retargeting that animates a target subject with the motion of a source subject. Our method is correspondence-free, requiring neither spatial correspondences between the source and target shapes nor temporal correspondences between different frames of the source motion. This allows to animate a target shape with arbitrary sequences of humans in motion, possibly captured using 4D acquisition platforms or consumer devices. Our method unifies the advantages of two existing lines of work, namely skeletal motion retargeting, which leverages long-term temporal context, and surface-based retargeting, which preserves surface details, by combining a geometry-aware deformation model with a skeleton-aware motion transfer approach. This allows to take into account long-term temporal context while accounting for surface details. During inference, our method runs online, i.e. input can be processed in a serial way, and retargeting is performed in a single forward pass per frame. Experiments show that including long-term temporal context during training improves the method's accuracy for skeletal motion and detail preservation. Furthermore, our method generalizes to unobserved motions and body shapes. We demonstrate that our method achieves state-of-the-art results on two test datasets and that it can be used to animate human models with the output of a multi-view acquisition platform. Code is available at \url{https://gitlab.inria.fr/rrekikdi/human-motion-retargeting2023}.
△ Less
Submitted 4 March, 2024; v1 submitted 1 February, 2023;
originally announced February 2023.
-
Automatic Discovery of Composite SPMD Partitioning Strategies in PartIR
Authors:
Sami Alabed,
Dominik Grewe,
Juliana Franco,
Bart Chrzaszcz,
Tom Natan,
Tamara Norman,
Norman A. Rink,
Dimitrios Vytiniotis,
Michael Schaarschmidt
Abstract:
Large neural network models are commonly trained through a combination of advanced parallelism strategies in a single program, multiple data (SPMD) paradigm. For example, training large transformer models requires combining data, model, and pipeline partitioning; and optimizer sharding techniques. However, identifying efficient combinations for many model architectures and accelerator systems requ…
▽ More
Large neural network models are commonly trained through a combination of advanced parallelism strategies in a single program, multiple data (SPMD) paradigm. For example, training large transformer models requires combining data, model, and pipeline partitioning; and optimizer sharding techniques. However, identifying efficient combinations for many model architectures and accelerator systems requires significant manual analysis. In this work, we present an automatic partitioner that identifies these combinations through a goal-oriented search. Our key findings are that a Monte Carlo Tree Search-based partitioner leveraging partition-specific compiler analysis directly into the search and guided goals matches expert-level strategies for various models.
△ Less
Submitted 7 October, 2022;
originally announced October 2022.
-
Representing motion as a sequence of latent primitives, a flexible approach for human motion modelling
Authors:
Mathieu Marsot,
Stefanie Wuhrer,
Jean-Sebastien Franco,
Anne Hélène Olivier
Abstract:
We propose a new representation of human body motion which encodes a full motion in a sequence of latent motion primitives. Recently, task generic motion priors have been introduced and propose a coherent representation of human motion based on a single latent code, with encouraging results for many tasks. Extending these methods to longer motion with various duration and framerate is all but stra…
▽ More
We propose a new representation of human body motion which encodes a full motion in a sequence of latent motion primitives. Recently, task generic motion priors have been introduced and propose a coherent representation of human motion based on a single latent code, with encouraging results for many tasks. Extending these methods to longer motion with various duration and framerate is all but straightforward as one latent code proves inefficient to encode longer term variability. Our hypothesis is that long motions are better represented as a succession of actions than in a single block. By leveraging a sequence-to-sequence architecture, we propose a model that simultaneously learns a temporal segmentation of motion and a prior on the motion segments. To provide flexibility with temporal resolution and motion duration, our representation is continuous in time and can be queried for any timestamp. We show experimentally that our method leads to a significant improvement over state-of-the-art motion priors on a spatio-temporal completion task on sparse pointclouds. Code will be made available upon publication.
△ Less
Submitted 1 September, 2022; v1 submitted 27 June, 2022;
originally announced June 2022.
-
A Survey of Honeypots and Honeynets for Internet of Things, Industrial Internet of Things, and Cyber-Physical Systems
Authors:
Javier Franco,
Ahmet Aris,
Berk Canberk,
A. Selcuk Uluagac
Abstract:
The Internet of Things (IoT), the Industrial Internet of Things (IIoT), and Cyber-Physical Systems (CPS) have become essential for our daily lives in contexts such as our homes, buildings, cities, health, transportation, manufacturing, infrastructure, and agriculture. However, they have become popular targets of attacks, due to their inherent limitations which create vulnerabilities. Honeypots and…
▽ More
The Internet of Things (IoT), the Industrial Internet of Things (IIoT), and Cyber-Physical Systems (CPS) have become essential for our daily lives in contexts such as our homes, buildings, cities, health, transportation, manufacturing, infrastructure, and agriculture. However, they have become popular targets of attacks, due to their inherent limitations which create vulnerabilities. Honeypots and honeynets can prove essential to understand and defend against attacks on IoT, IIoT, and CPS environments by attracting attackers and deceiving them into thinking that they have gained access to the real systems. Honeypots and honeynets can complement other security solutions (i.e., firewalls, Intrusion Detection Systems - IDS) to form a strong defense against malicious entities. This paper provides a comprehensive survey of the research that has been carried out on honeypots and honeynets for IoT, IIoT, and CPS. It provides a taxonomy and extensive analysis of the existing honeypots and honeynets, states key design factors for the state-of-the-art honeypot/honeynet research and outlines open issues for future honeypots and honeynets for IoT, IIoT, and CPS environments.
△ Less
Submitted 4 August, 2021;
originally announced August 2021.
-
A Literature Review on Blockchain-enabled Security and Operation of Cyber-Physical Systems
Authors:
Alvi Ataur Khalil,
Javier Franco,
Imtiaz Parvez,
Selcuk Uluagac,
Mohammad Ashiqur Rahman
Abstract:
Blockchain has become a key technology in a plethora of application domains owing to its decentralized public nature. The cyber-physical systems (CPS) is one of the prominent application domains that leverage blockchain for myriad operations, where the Internet of Things (IoT) is utilized for data collection. Although some of the CPS problems can be solved by simply adopting blockchain for its sec…
▽ More
Blockchain has become a key technology in a plethora of application domains owing to its decentralized public nature. The cyber-physical systems (CPS) is one of the prominent application domains that leverage blockchain for myriad operations, where the Internet of Things (IoT) is utilized for data collection. Although some of the CPS problems can be solved by simply adopting blockchain for its secure and distributed nature, others require complex considerations for overcoming blockchain-imposed limitations while maintaining the core aspect of CPS. Even though a number of studies focus on either the utilization of blockchains for different CPS applications or the blockchain-enabled security of CPS, there is no comprehensive survey including both perspectives together. To fill this gap, we present a comprehensive overview of contemporary advancement in using blockchain for enhancing different CPS operations as well as improving CPS security. To the best of our knowledge, this is the first paper that presents an in-depth review of research on blockchain-enabled CPS operation and security.
△ Less
Submitted 16 July, 2021;
originally announced July 2021.
-
A structured latent space for human body motion generation
Authors:
Mathieu Marsot,
Stefanie Wuhrer,
Jean-Sebastien Franco,
Stephane Durocher
Abstract:
We propose a framework to learn a structured latent space to represent 4D human body motion, where each latent vector encodes a full motion of the whole 3D human shape. On one hand several data-driven skeletal animation models exist proposing motion spaces of temporally dense motion signals, but based on geometrically sparse kinematic representations. On the other hand many methods exist to build…
▽ More
We propose a framework to learn a structured latent space to represent 4D human body motion, where each latent vector encodes a full motion of the whole 3D human shape. On one hand several data-driven skeletal animation models exist proposing motion spaces of temporally dense motion signals, but based on geometrically sparse kinematic representations. On the other hand many methods exist to build shape spaces of dense 3D geometry, but for static frames. We bring together both concepts, proposing a motion space that is dense both temporally and geometrically. Once trained, our model generates a multi-frame sequence of dense 3D meshes based on a single point in a low-dimensional latent space. This latent space is built to be structured, such that similar motions form clusters. It also embeds variations of duration in the latent vector, allowing semantically close sequences that differ only by temporal unfolding to share similar latent vectors. We demonstrate experimentally the structural properties of our latent space, and show it can be used to generate plausible interpolations between different actions. We also apply our model to 4D human motion completion, showing its promising abilities to learn spatio-temporal features of human motion.
△ Less
Submitted 1 September, 2022; v1 submitted 7 June, 2021;
originally announced June 2021.
-
Fast and Memory-Efficient Neural Code Completion
Authors:
Alexey Svyatkovskiy,
Sebastian Lee,
Anna Hadjitofi,
Maik Riechert,
Juliana Franco,
Miltiadis Allamanis
Abstract:
Code completion is one of the most widely used features of modern integrated development environments (IDEs). While deep learning has made significant progress in the statistical prediction of source code, state-of-the-art neural network models consume hundreds of megabytes of memory, bloating the development environment. We address this in two steps: first we present a modular neural framework fo…
▽ More
Code completion is one of the most widely used features of modern integrated development environments (IDEs). While deep learning has made significant progress in the statistical prediction of source code, state-of-the-art neural network models consume hundreds of megabytes of memory, bloating the development environment. We address this in two steps: first we present a modular neural framework for code completion. This allows us to explore the design space and evaluate different techniques. Second, within this framework we design a novel reranking neural completion model that combines static analysis with granular token encodings. The best neural reranking model consumes just 6 MB of RAM, - 19x less than previous models - computes a single completion in 8 ms, and achieves 90% accuracy in its top five suggestions.
△ Less
Submitted 16 March, 2021; v1 submitted 28 April, 2020;
originally announced April 2020.
-
Moulding Humans: Non-parametric 3D Human Shape Estimation from Single Images
Authors:
Valentin Gabeur,
Jean-Sebastien Franco,
Xavier Martin,
Cordelia Schmid,
Gregory Rogez
Abstract:
In this paper, we tackle the problem of 3D human shape estimation from single RGB images. While the recent progress in convolutional neural networks has allowed impressive results for 3D human pose estimation, estimating the full 3D shape of a person is still an open issue. Model-based approaches can output precise meshes of naked under-cloth human bodies but fail to estimate details and un-modell…
▽ More
In this paper, we tackle the problem of 3D human shape estimation from single RGB images. While the recent progress in convolutional neural networks has allowed impressive results for 3D human pose estimation, estimating the full 3D shape of a person is still an open issue. Model-based approaches can output precise meshes of naked under-cloth human bodies but fail to estimate details and un-modelled elements such as hair or clothing. On the other hand, non-parametric volumetric approaches can potentially estimate complete shapes but, in practice, they are limited by the resolution of the output grid and cannot produce detailed estimates. In this work, we propose a non-parametric approach that employs a double depth map to represent the 3D shape of a person: a visible depth map and a "hidden" depth map are estimated and combined, to reconstruct the human 3D shape as done with a "mould". This representation through 2D depth maps allows a higher resolution output with a much lower dimension than voxel-based volumetric representations. Additionally, our fully derivable depth-based model allows us to efficiently incorporate a discriminator in an adversarial fashion to improve the accuracy and "humanness" of the 3D output. We train and quantitatively validate our approach on SURREAL and on 3D-HUMANS, a new photorealistic dataset made of semi-synthetic in-house videos annotated with 3D ground truth surfaces.
△ Less
Submitted 1 August, 2019;
originally announced August 2019.
-
Safely Abstracting Memory Layouts
Authors:
Juliana Franco,
Alexandros Tasos,
Sophia Drossopoulou,
Tobias Wrigstad,
Susan Eisenbach
Abstract:
Modern architectures require applications to make effective use of caches to achieve high performance and hide memory latency. This in turn requires careful consideration of placement of data in memory to exploit spatial locality, leverage hardware prefetching and conserve memory bandwidth. In unmanaged languages like C++, memory optimisations are common, but at the cost of losing object abstracti…
▽ More
Modern architectures require applications to make effective use of caches to achieve high performance and hide memory latency. This in turn requires careful consideration of placement of data in memory to exploit spatial locality, leverage hardware prefetching and conserve memory bandwidth. In unmanaged languages like C++, memory optimisations are common, but at the cost of losing object abstraction and memory safety. In managed languages like Java and C#, the abstract view of memory and proliferation of moving compacting garbage collection does not provide enough control over placement and layout.
We have proposed SHAPES, a type-driven abstract placement specification that can be integrated with object-oriented languages to enable memory optimisations. SHAPES preserves both memory and object abstraction. In this paper, we formally specify the SHAPES semantics and describe its memory safety model.
△ Less
Submitted 23 January, 2019;
originally announced January 2019.
-
QuickCSG: Fast Arbitrary Boolean Combinations of N Solids
Authors:
Matthijs Douze,
Jean-Sébastien Franco,
Bruno Raffin
Abstract:
QuickCSG computes the result for general N-polyhedron boolean expressions without an intermediate tree of solids. We propose a vertex-centric view of the problem, which simplifies the identification of final geometric contributions, and facilitates its spatial decomposition. The problem is then cast in a single KD-tree exploration, geared toward the result by early pruning of any region of space n…
▽ More
QuickCSG computes the result for general N-polyhedron boolean expressions without an intermediate tree of solids. We propose a vertex-centric view of the problem, which simplifies the identification of final geometric contributions, and facilitates its spatial decomposition. The problem is then cast in a single KD-tree exploration, geared toward the result by early pruning of any region of space not contributing to the final surface. We assume strong regularity properties on the input meshes and that they are in general position. This simplifying assumption, in combination with our vertex-centric approach, improves the speed of the approach. Complemented with a task-stealing parallelization, the algorithm achieves breakthrough performance, one to two orders of magnitude speedups with respect to state-of-the-art CPU algorithms, on boolean operations over two to dozens of polyhedra. The algorithm also outperforms GPU implementations with approximate discretizations, while producing an output without redundant facets. Despite the restrictive assumptions on the input, we show the usefulness of QuickCSG for applications with large CSG problems and strong temporal constraints, e.g. modeling for 3D printers, reconstruction from visual hulls and collision detection.
△ Less
Submitted 5 June, 2017;
originally announced June 2017.
-
Behavioural types for non-uniform memory accesses
Authors:
Juliana Franco,
Sophia Drossopoulou
Abstract:
Concurrent programs executing on NUMA architectures consist of concurrent entities (e.g. threads, actors) and data placed on different nodes. Execution of these concurrent entities often reads or updates states from remote nodes. The performance of such systems depends on the extent to which the concurrent entities can be executing in parallel, and on the amount of the remote reads and wri…
▽ More
Concurrent programs executing on NUMA architectures consist of concurrent entities (e.g. threads, actors) and data placed on different nodes. Execution of these concurrent entities often reads or updates states from remote nodes. The performance of such systems depends on the extent to which the concurrent entities can be executing in parallel, and on the amount of the remote reads and writes.
We consider an actor-based object oriented language, and propose a type system which expresses the topology of the program (the placement of the actors and data on the nodes), and an effect system which characterises remote reads and writes (in terms of which node reads/writes from which other nodes). We use a variant of ownership types for the topology, and a combination of behavioural and ownership types for the effect system.
△ Less
Submitted 10 February, 2016;
originally announced February 2016.
-
Shape Animation with Combined Captured and Simulated Dynamics
Authors:
Benjamin Allain,
Li Wang,
Jean-Sebastien Franco,
Franck Hetroy,
Edmond Boyer
Abstract:
We present a novel volumetric animation generation framework to create new types of animations from raw 3D surface or point cloud sequence of captured real performances. The framework considers as input time incoherent 3D observations of a moving shape, and is thus particularly suitable for the output of performance capture platforms. In our system, a suitable virtual representation of the actor i…
▽ More
We present a novel volumetric animation generation framework to create new types of animations from raw 3D surface or point cloud sequence of captured real performances. The framework considers as input time incoherent 3D observations of a moving shape, and is thus particularly suitable for the output of performance capture platforms. In our system, a suitable virtual representation of the actor is built from real captures that allows seamless combination and simulation with virtual external forces and objects, in which the original captured actor can be reshaped, disassembled or reassembled from user-specified virtual physics. Instead of using the dominant surface-based geometric representation of the capture, which is less suitable for volumetric effects, our pipeline exploits Centroidal Voronoi tessellation decompositions as unified volumetric representation of the real captured actor, which we show can be used seamlessly as a building block for all processing stages, from capture and tracking to virtual physic simulation. The representation makes no human specific assumption and can be used to capture and re-simulate the actor with props or other moving scenery elements. We demonstrate the potential of this pipeline for virtual reanimation of a real captured event with various unprecedented volumetric visual effects, such as volumetric distortion, erosion, morphing, gravity pull, or collisions.
△ Less
Submitted 6 January, 2016;
originally announced January 2016.
-
A unified approach for multi-object triangulation, tracking and camera calibration
Authors:
Jeremie Houssineau,
Daniel Clark,
Spela Ivekovic,
Chee Sing Lee,
Jose Franco
Abstract:
Object triangulation, 3-D object tracking, feature correspondence, and camera calibration are key problems for estimation from camera networks. This paper addresses these problems within a unified Bayesian framework for joint multi-object tracking and sensor registration. Given that using standard filtering approaches for state estimation from cameras is problematic, an alternative parametrisation…
▽ More
Object triangulation, 3-D object tracking, feature correspondence, and camera calibration are key problems for estimation from camera networks. This paper addresses these problems within a unified Bayesian framework for joint multi-object tracking and sensor registration. Given that using standard filtering approaches for state estimation from cameras is problematic, an alternative parametrisation is exploited, called disparity space. The disparity space-based approach for triangulation and object tracking is shown to be more effective than non-linear versions of the Kalman filter and particle filtering for non-rectified cameras. The approach for feature correspondence is based on the Probability Hypothesis Density (PHD) filter, and hence inherits the ability to update without explicit measurement association, to initiate new targets, and to discriminate between target and clutter. The PHD filtering approach then forms the basis of a camera calibration method from static or moving objects. Results are shown on simulated data.
△ Less
Submitted 9 October, 2014;
originally announced October 2014.
-
The Unit-Demand Envy-Free Pricing Problem
Authors:
Cristina G. Fernandes,
Carlos E. Ferreira,
Álvaro J. P. Franco,
Rafael C. S. Schouery
Abstract:
We consider the unit-demand envy-free pricing problem, which is a unit-demand auction where each bidder receives an item that maximizes his utility, and the goal is to maximize the auctioneer's profit. This problem is NP-hard and unlikely to be in APX. We present four new MIP formulations for it and experimentally compare them to a previous one due to Shioda, Tunçel, and Myklebust. We describe thr…
▽ More
We consider the unit-demand envy-free pricing problem, which is a unit-demand auction where each bidder receives an item that maximizes his utility, and the goal is to maximize the auctioneer's profit. This problem is NP-hard and unlikely to be in APX. We present four new MIP formulations for it and experimentally compare them to a previous one due to Shioda, Tunçel, and Myklebust. We describe three models to generate different random instances for general unit-demand auctions, that we designed for the computational experiments. Each model has a nice economic interpretation. Aiming approximation results, we consider the variant of the problem where the item prices are restricted to be chosen from a geometric series, and prove that an optimal solution for this variant has value that is a fraction (depending on the series used) of the optimal value of the original problem. So this variant is also unlikely to be in APX.
△ Less
Submitted 30 September, 2013;
originally announced October 2013.
-
Algorithms for Junctions in Directed Acyclic Graphs
Authors:
Carlos Eduardo Ferreira,
Álvaro Junio Pereira Franco
Abstract:
Given a pair of distinct vertices u, v in a graph G, we say that s is a junction of u, v if there are in G internally vertex disjoint directed paths from s to u and from s to v. We show how to characterize junctions in directed acyclic graphs. We also consider the two problems in the following and derive efficient algorithms to solve them. Given a directed acyclic graph G and a vertex s in G, how…
▽ More
Given a pair of distinct vertices u, v in a graph G, we say that s is a junction of u, v if there are in G internally vertex disjoint directed paths from s to u and from s to v. We show how to characterize junctions in directed acyclic graphs. We also consider the two problems in the following and derive efficient algorithms to solve them. Given a directed acyclic graph G and a vertex s in G, how can we find all pairs of vertices of G such that s is a junction of them? And given a directed acyclic graph G and k pairs of vertices of G, how can we preprocess G such that all junctions of k given pairs of vertices could be listed quickly? All junctions of k pairs problem arises in an application in Anthropology and we apply our algorithm to find such junctions on kinship networks of some brazilian indian ethnic groups.
△ Less
Submitted 13 April, 2012;
originally announced April 2012.
-
Improved Complexity Bound of Vertex Cover for Low degree Graph
Authors:
Weiya Yue,
John Franco,
Weiwei Cao
Abstract:
In this paper, we use a new method to decrease the parameterized complexity bound for finding the minimum vertex cover of connected max-degree-3 undirected graphs. The key operation of this method is reduction of the size of a particular subset of edges which we introduce in this paper and is called as "real-cycle" subset. Using "real-cycle" reductions alone we compute a complexity bound…
▽ More
In this paper, we use a new method to decrease the parameterized complexity bound for finding the minimum vertex cover of connected max-degree-3 undirected graphs. The key operation of this method is reduction of the size of a particular subset of edges which we introduce in this paper and is called as "real-cycle" subset. Using "real-cycle" reductions alone we compute a complexity bound $O(1.15855^k)$ where $k$ is size of the optimal vertex cover. Combined with other techniques, the complexity bound can be further improved to be $O(1.1504^k)$. This is currently the best complexity bound.
△ Less
Submitted 14 October, 2010;
originally announced October 2010.
-
A Radar-Shaped Statistic for Testing and Visualizing Uniformity Properties in Computer Experiments
Authors:
Jessica Franco,
Laurent Carraro,
Olivier Roustant,
Astrid Jourdan
Abstract:
In the study of computer codes, filling space as uniformly as possible is important to describe the complexity of the investigated phenomenon. However, this property is not conserved by reducing the dimension. Some numeric experiment designs are conceived in this sense as Latin hypercubes or orthogonal arrays, but they consider only the projections onto the axes or the coordinate planes. In this…
▽ More
In the study of computer codes, filling space as uniformly as possible is important to describe the complexity of the investigated phenomenon. However, this property is not conserved by reducing the dimension. Some numeric experiment designs are conceived in this sense as Latin hypercubes or orthogonal arrays, but they consider only the projections onto the axes or the coordinate planes. In this article we introduce a statistic which allows studying the good distribution of points according to all 1-dimensional projections. By angularly scanning the domain, we obtain a radar type representation, allowing the uniformity defects of a design to be identified with respect to its projections onto straight lines. The advantages of this new tool are demonstrated on usual examples of space-filling designs (SFD) and a global statistic independent of the angle of rotation is studied.
△ Less
Submitted 15 February, 2008;
originally announced February 2008.