-
Rethinking Attention Module Design for Point Cloud Analysis
Authors:
Chengzhi Wu,
Kaige Wang,
Zeyun Zhong,
Hao Fu,
Junwei Zheng,
Jiaming Zhang,
Julius Pfrommer,
Jürgen Beyerer
Abstract:
In recent years, there have been significant advancements in applying attention mechanisms to point cloud analysis. However, attention module variants featured in various research papers often operate under diverse settings and tasks, incorporating potential training strategies. This heterogeneity poses challenges in establishing a fair comparison among these attention module variants. In this pap…
▽ More
In recent years, there have been significant advancements in applying attention mechanisms to point cloud analysis. However, attention module variants featured in various research papers often operate under diverse settings and tasks, incorporating potential training strategies. This heterogeneity poses challenges in establishing a fair comparison among these attention module variants. In this paper, we address this issue by rethinking and exploring attention module design within a consistent base framework and settings. Both global-based and local-based attention methods are studied, with a focus on the selection basis and scales of neighbors for local-based attention. Different combinations of aggregated local features and computation methods for attention scores are evaluated, ranging from the initial addition/concatenation-based approach to the widely adopted dot product-based method and the recently proposed vector attention technique. Various position encoding methods are also investigated. Our extensive experimental analysis reveals that there is no universally optimal design across diverse point cloud tasks. Instead, drawing from best practices, we propose tailored attention modules for specific tasks, leading to superior performance on point cloud classification and segmentation benchmarks.
△ Less
Submitted 27 July, 2024;
originally announced July 2024.
-
Sorting multibay block stacking storage systems
Authors:
Jakob Pfrommer,
Thomas Bömer,
Daniyar Akizhanov,
Anne Meyer
Abstract:
Autonomous mobile robots (AMRs) are increasingly used to automate operations in intralogistics. One crucial feature of AMRs is their availability, allowing them to operate 24/7. This work addresses the multibay unit load pre-marshalling problem, which extends pre-marshalling from a single bay to larger warehouse configurations with multiple bays. Pre-marshalling leverages off-peak time intervals t…
▽ More
Autonomous mobile robots (AMRs) are increasingly used to automate operations in intralogistics. One crucial feature of AMRs is their availability, allowing them to operate 24/7. This work addresses the multibay unit load pre-marshalling problem, which extends pre-marshalling from a single bay to larger warehouse configurations with multiple bays. Pre-marshalling leverages off-peak time intervals to sort a block stacking warehouse in anticipation of future orders. These larger warehouse configurations require not only the minimization of the number of moves but also the consideration of distance or time when making sorting decisions. Our proposed solution for the multibay unit load pre-marshalling problem is based on our two-step approach that first determines the access direction for each stack and then finds a sequence of moves to sort the warehouse. In addition to adapting the existing approach that integrates a network flow model and an extended A* algorithm, we additionally present an exact constraint programming approach for the second stage of the problem-solving process. The results demonstrate that the presented solution approach effectively enhances the access time of unit loads and reduces the sorting effort for block stacking warehouses with multiple bays.
△ Less
Submitted 8 May, 2024;
originally announced May 2024.
-
Attention-based Part Assembly for 3D Volumetric Shape Modeling
Authors:
Chengzhi Wu,
Junwei Zheng,
Julius Pfrommer,
Jürgen Beyerer
Abstract:
Modeling a 3D volumetric shape as an assembly of decomposed shape parts is much more challenging, but semantically more valuable than direct reconstruction from a full shape representation. The neural network needs to implicitly learn part relations coherently, which is typically performed by dedicated network layers that can generate transformation matrices for each part. In this paper, we propos…
▽ More
Modeling a 3D volumetric shape as an assembly of decomposed shape parts is much more challenging, but semantically more valuable than direct reconstruction from a full shape representation. The neural network needs to implicitly learn part relations coherently, which is typically performed by dedicated network layers that can generate transformation matrices for each part. In this paper, we propose a VoxAttention network architecture for attention-based part assembly. We further propose a variant of using channel-wise part attention and show the advantages of this approach. Experimental results show that our method outperforms most state-of-the-art methods for the part relation-aware 3D shape modeling task.
△ Less
Submitted 17 April, 2023;
originally announced April 2023.
-
Attention-based Point Cloud Edge Sampling
Authors:
Chengzhi Wu,
Junwei Zheng,
Julius Pfrommer,
Jürgen Beyerer
Abstract:
Point cloud sampling is a less explored research topic for this data representation. The most commonly used sampling methods are still classical random sampling and farthest point sampling. With the development of neural networks, various methods have been proposed to sample point clouds in a task-based learning manner. However, these methods are mostly generative-based, rather than selecting poin…
▽ More
Point cloud sampling is a less explored research topic for this data representation. The most commonly used sampling methods are still classical random sampling and farthest point sampling. With the development of neural networks, various methods have been proposed to sample point clouds in a task-based learning manner. However, these methods are mostly generative-based, rather than selecting points directly using mathematical statistics. Inspired by the Canny edge detection algorithm for images and with the help of the attention mechanism, this paper proposes a non-generative Attention-based Point cloud Edge Sampling method (APES), which captures salient points in the point cloud outline. Both qualitative and quantitative experimental results show the superior performance of our sampling method on common benchmark tasks.
△ Less
Submitted 26 March, 2023; v1 submitted 28 February, 2023;
originally announced February 2023.
-
Sim2real Transfer Learning for Point Cloud Segmentation: An Industrial Application Case on Autonomous Disassembly
Authors:
Chengzhi Wu,
Xuelei Bi,
Julius Pfrommer,
Alexander Cebulla,
Simon Mangold,
Jürgen Beyerer
Abstract:
On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data,…
▽ More
On robotics computer vision tasks, generating and annotating large amounts of data from real-world for the use of deep learning-based approaches is often difficult or even impossible. A common strategy for solving this problem is to apply simulation-to-reality (sim2real) approaches with the help of simulated scenes. While the majority of current robotics vision sim2real work focuses on image data, we present an industrial application case that uses sim2real transfer learning for point cloud data. We provide insights on how to generate and process synthetic point cloud data in order to achieve better performance when the learned model is transferred to real-world data. The issue of imbalanced learning is investigated using multiple strategies. A novel patch-based attention network is proposed additionally to tackle this problem.
△ Less
Submitted 12 January, 2023;
originally announced January 2023.
-
MotorFactory: A Blender Add-on for Large Dataset Generation of Small Electric Motors
Authors:
Chengzhi Wu,
Kanran Zhou,
Jan-Philipp Kaiser,
Norbert Mitschke,
Jan-Felix Klein,
Julius Pfrommer,
Jürgen Beyerer,
Gisela Lanza,
Michael Heizmann,
Kai Furmans
Abstract:
To enable automatic disassembly of different product types with uncertain conditions and degrees of wear in remanufacturing, agile production systems that can adapt dynamically to changing requirements are needed. Machine learning algorithms can be employed due to their generalization capabilities of learning from various types and variants of products. However, in reality, datasets with a diversi…
▽ More
To enable automatic disassembly of different product types with uncertain conditions and degrees of wear in remanufacturing, agile production systems that can adapt dynamically to changing requirements are needed. Machine learning algorithms can be employed due to their generalization capabilities of learning from various types and variants of products. However, in reality, datasets with a diversity of samples that can be used to train models are difficult to obtain in the initial period. This may cause bad performances when the system tries to adapt to new unseen input data in the future. In order to generate large datasets for different learning purposes, in our project, we present a Blender add-on named MotorFactory to generate customized mesh models of various motor instances. MotorFactory allows to create mesh models which, complemented with additional add-ons, can be further used to create synthetic RGB images, depth images, normal images, segmentation ground truth masks, and 3D point cloud datasets with point-wise semantic labels. The created synthetic datasets may be used for various tasks including motor type classification, object detection for decentralized material transfer tasks, part segmentation for disassembly and handling tasks, or even reinforcement learning-based robotics control or view-planning.
△ Less
Submitted 11 January, 2023;
originally announced January 2023.
-
SynMotor: A Benchmark Suite for Object Attribute Regression and Multi-task Learning
Authors:
Chengzhi Wu,
Linxi Qiu,
Kanran Zhou,
Julius Pfrommer,
Jürgen Beyerer
Abstract:
In this paper, we develop a novel benchmark suite including both a 2D synthetic image dataset and a 3D synthetic point cloud dataset. Our work is a sub-task in the framework of a remanufacturing project, in which small electric motors are used as fundamental objects. Apart from the given detection, classification, and segmentation annotations, the key objects also have multiple learnable attribute…
▽ More
In this paper, we develop a novel benchmark suite including both a 2D synthetic image dataset and a 3D synthetic point cloud dataset. Our work is a sub-task in the framework of a remanufacturing project, in which small electric motors are used as fundamental objects. Apart from the given detection, classification, and segmentation annotations, the key objects also have multiple learnable attributes with ground truth provided. This benchmark can be used for computer vision tasks including 2D/3D detection, classification, segmentation, and multi-attribute learning. It is worth mentioning that most attributes of the motors are quantified as continuously variable rather than binary, which makes our benchmark well-suited for the less explored regression tasks. In addition, appropriate evaluation metrics are adopted or developed for each task and promising baseline results are provided. We hope this benchmark can stimulate more research efforts on the sub-domain of object attribute learning and multi-task learning in the future.
△ Less
Submitted 11 January, 2023;
originally announced January 2023.
-
Object Detection in 3D Point Clouds via Local Correlation-Aware Point Embedding
Authors:
Chengzhi Wu,
Julius Pfrommer,
Jürgen Beyerer,
Kangning Li,
Boris Neubert
Abstract:
We present an improved approach for 3D object detection in point cloud data based on the Frustum PointNet (F-PointNet). Compared to the original F-PointNet, our newly proposed method considers the point neighborhood when computing point features. The newly introduced local neighborhood embedding operation mimics the convolutional operations in 2D neural networks. Thus features of each point are no…
▽ More
We present an improved approach for 3D object detection in point cloud data based on the Frustum PointNet (F-PointNet). Compared to the original F-PointNet, our newly proposed method considers the point neighborhood when computing point features. The newly introduced local neighborhood embedding operation mimics the convolutional operations in 2D neural networks. Thus features of each point are not only computed with the features of its own or of the whole point cloud but also computed especially with respect to the features of its neighbors. Experiments show that our proposed method achieves better performance than the F-Pointnet baseline on 3D object detection tasks.
△ Less
Submitted 11 January, 2023;
originally announced January 2023.
-
Generative-Contrastive Learning for Self-Supervised Latent Representations of 3D Shapes from Multi-Modal Euclidean Input
Authors:
Chengzhi Wu,
Julius Pfrommer,
Mingyuan Zhou,
Jürgen Beyerer
Abstract:
We propose a combined generative and contrastive neural architecture for learning latent representations of 3D volumetric shapes. The architecture uses two encoder branches for voxel grids and multi-view images from the same underlying shape. The main idea is to combine a contrastive loss between the resulting latent representations with an additional reconstruction loss. That helps to avoid colla…
▽ More
We propose a combined generative and contrastive neural architecture for learning latent representations of 3D volumetric shapes. The architecture uses two encoder branches for voxel grids and multi-view images from the same underlying shape. The main idea is to combine a contrastive loss between the resulting latent representations with an additional reconstruction loss. That helps to avoid collapsing the latent representations as a trivial solution for minimizing the contrastive loss. A novel switching scheme is used to cross-train two encoders with a shared decoder. The switching scheme also enables the stop gradient operation on a random branch. Further classification experiments show that the latent representations learned with our self-supervised method integrate more useful information from the additional input data implicitly, thus leading to better reconstruction and classification performance.
△ Less
Submitted 11 January, 2023;
originally announced January 2023.
-
Solving the unit-load pre-marshalling problem in block stacking storage systems with multiple access directions
Authors:
Jakob Pfrommer,
Anne Meyer,
Kevin Tierney
Abstract:
Block stacking storage systems are highly adaptable warehouse systems with low investment costs. With multiple, deep lanes they can achieve high storage densities, but accessing some unit loads can be time-consuming. The unit-load pre-marshalling problem sorts the unit loads in a block stacking storage system in off-peak time periods to prepare for upcoming orders. The goal is to find a minimum nu…
▽ More
Block stacking storage systems are highly adaptable warehouse systems with low investment costs. With multiple, deep lanes they can achieve high storage densities, but accessing some unit loads can be time-consuming. The unit-load pre-marshalling problem sorts the unit loads in a block stacking storage system in off-peak time periods to prepare for upcoming orders. The goal is to find a minimum number of unit-load moves needed to sequence a storage bay in ascending order based on the retrieval priority group of each unit load. In this paper, we present two solution approaches for determining the minimum number of unit-load moves. We show that for storage bays with one access direction, it is possible to adapt existing, optimal tree search procedures and lower bound heuristics from the container pre-marshalling problem. For multiple access directions, we develop a novel, two-step solution approach based on a network flow model and an A* algorithm with an adapted lower bound that is applicable in all scenarios. We further analyze the performance of the presented solutions in computational experiments for randomly generated problem instances and show that multiple access directions greatly reduce both the total access time of unit loads and the required sorting effort.
△ Less
Submitted 19 July, 2022;
originally announced July 2022.
-
Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems
Authors:
Laura von Rueden,
Sebastian Mayer,
Katharina Beckh,
Bogdan Georgiev,
Sven Giesselbach,
Raoul Heese,
Birgit Kirsch,
Julius Pfrommer,
Annika Pick,
Rajkumar Ramamurthy,
Michal Walczak,
Jochen Garcke,
Christian Bauckhage,
Jannis Schuecker
Abstract:
Despite its great success, machine learning can have its limits when dealing with insufficient training data. A potential solution is the additional integration of prior knowledge into the training process which leads to the notion of informed machine learning. In this paper, we present a structured overview of various approaches in this field. We provide a definition and propose a concept for inf…
▽ More
Despite its great success, machine learning can have its limits when dealing with insufficient training data. A potential solution is the additional integration of prior knowledge into the training process which leads to the notion of informed machine learning. In this paper, we present a structured overview of various approaches in this field. We provide a definition and propose a concept for informed machine learning which illustrates its building blocks and distinguishes it from conventional machine learning. We introduce a taxonomy that serves as a classification framework for informed machine learning approaches. It considers the source of knowledge, its representation, and its integration into the machine learning pipeline. Based on this taxonomy, we survey related research and describe how different knowledge representations such as algebraic equations, logic rules, or simulation results can be used in learning systems. This evaluation of numerous papers on the basis of our taxonomy uncovers key methods in the field of informed machine learning.
△ Less
Submitted 28 May, 2021; v1 submitted 29 March, 2019;
originally announced March 2019.