A Scaling Law for Synthetic-to-Real Transfer: How Much Is Your Pre-training Effective?
Authors:
Hiroaki Mikami,
Kenji Fukumizu,
Shogo Murai,
Shuji Suzuki,
Yuta Kikuchi,
Taiji Suzuki,
Shin-ichi Maeda,
Kohei Hayashi
Abstract:
Synthetic-to-real transfer learning is a framework in which a synthetically generated dataset is used to pre-train a model to improve its performance on real vision tasks. The most significant advantage of using synthetic images is that the ground-truth labels are automatically available, enabling unlimited expansion of the data size without human cost. However, synthetic data may have a huge doma…
▽ More
Synthetic-to-real transfer learning is a framework in which a synthetically generated dataset is used to pre-train a model to improve its performance on real vision tasks. The most significant advantage of using synthetic images is that the ground-truth labels are automatically available, enabling unlimited expansion of the data size without human cost. However, synthetic data may have a huge domain gap, in which case increasing the data size does not improve the performance. How can we know that? In this study, we derive a simple scaling law that predicts the performance from the amount of pre-training data. By estimating the parameters of the law, we can judge whether we should increase the data or change the setting of image synthesis. Further, we analyze the theory of transfer learning by considering learning dynamics and confirm that the derived generalization bound is consistent with our empirical findings. We empirically validated our scaling law on various experimental settings of benchmark tasks, model sizes, and complexities of synthetic images.
△ Less
Submitted 8 October, 2021; v1 submitted 24 August, 2021;
originally announced August 2021.
Collection of abstracts of the Workshop on Triangulations in Geometry and Topology at CG Week 2014 in Kyoto
Authors:
Jonathan Spreer,
Uli Wagner,
Benjamin A. Burton,
Satoshi Murai,
Eric Sedgwick,
Henry Segerman
Abstract:
This workshop about triangulations of manifolds in computational geometry and topology was held at the 2014 CG-Week in Kyoto, Japan.
It focussed on computational and combinatorial questions regarding triangulations, with the goal of bringing together researchers working on various aspects of triangulations and of fostering a closer collaboration within the computational geometry and topology com…
▽ More
This workshop about triangulations of manifolds in computational geometry and topology was held at the 2014 CG-Week in Kyoto, Japan.
It focussed on computational and combinatorial questions regarding triangulations, with the goal of bringing together researchers working on various aspects of triangulations and of fostering a closer collaboration within the computational geometry and topology community.
Triangulations are highly suitable for computations due to their clear combinatorial structure. As a consequence, they have been successfully employed in discrete algorithms to solve purely theoretical problems in a broad variety of mathematical research areas (knot theory, polytope theory, 2- and 3-manifold topology, geometry, and others). However, due to the large variety of applications, requirements vary from field to field and thus different types of triangulations, different tools, and different frameworks are used in different areas of research. This is why today closely related research areas are sometimes largely disjoint leaving potential reciprocal benefits unused.
To address these potentials a workshop on Triangulations was held at Oberwolfach Research Institute in 2012. Since then many new collaborations between researchers of different mathematical communities have been established. Regarding the computational geometry community, the theory of manifolds continues to contribute to advances in more applied areas of the field. Many researchers are interested in fundamental mathematical research about triangulations and thus will benefit from a broad set of knowledge about different research areas using different techniques.
We hope that this workshop brought together researchers from many different fields of computational geometry to have fruitful discussions which will lead to new interdisciplinary collaborations and solutions.
△ Less
Submitted 2 June, 2014;
originally announced June 2014.
