-
Proper losses regret at least 1/2-order
Authors:
Han Bao,
Asuka Takatsu
Abstract:
A fundamental challenge in machine learning is the choice of a loss as it characterizes our learning task, is minimized in the training phase, and serves as an evaluation criterion for estimators. Proper losses are commonly chosen, ensuring minimizers of the full risk match the true probability vector. Estimators induced from a proper loss are widely used to construct forecasters for downstream ta…
▽ More
A fundamental challenge in machine learning is the choice of a loss as it characterizes our learning task, is minimized in the training phase, and serves as an evaluation criterion for estimators. Proper losses are commonly chosen, ensuring minimizers of the full risk match the true probability vector. Estimators induced from a proper loss are widely used to construct forecasters for downstream tasks such as classification and ranking. In this procedure, how does the forecaster based on the obtained estimator perform well under a given downstream task? This question is substantially relevant to the behavior of the $p$-norm between the estimated and true probability vectors when the estimator is updated. In the proper loss framework, the suboptimality of the estimated probability vector from the true probability vector is measured by a surrogate regret. First, we analyze a surrogate regret and show that the strict properness of a loss is necessary and sufficient to establish a non-vacuous surrogate regret bound. Second, we solve an important open question that the order of convergence in p-norm cannot be faster than the $1/2$-order of surrogate regrets for a broad class of strictly proper losses. This implies that strongly proper losses entail the optimal convergence rate.
△ Less
Submitted 14 July, 2024;
originally announced July 2024.
-
Equal area partitions of the sphere with diameter bounds, via optimal transport
Authors:
Jun Kitagawa,
Asuka Takatsu
Abstract:
We prove existence of equal area partitions of the unit sphere via optimal transport methods, accompanied by diameter bounds written in terms of Monge--Kantorovich distances. This can be used to obtain bounds on the expectation of the maximum diameter of partition sets, when points are uniformly sampled from the sphere. An application to the computation of sliced Monge--Kantorovich distances is al…
▽ More
We prove existence of equal area partitions of the unit sphere via optimal transport methods, accompanied by diameter bounds written in terms of Monge--Kantorovich distances. This can be used to obtain bounds on the expectation of the maximum diameter of partition sets, when points are uniformly sampled from the sphere. An application to the computation of sliced Monge--Kantorovich distances is also presented.
△ Less
Submitted 28 June, 2023;
originally announced June 2023.
-
Change the coefficients of conditional entropies in extensivity
Authors:
Asuka Takatsu
Abstract:
The Boltzmann--Gibbs entropy is a functional on the space of probability measures. When a state space is countable, one characterization of the Boltzmann--Gibbs entropy is given by the Shannon--Khinchin axioms, which consist of continuity, maximality, expandability and extensivity. Among these four properties, the extensivity is generalized in various ways. The extensivity of a functional is inter…
▽ More
The Boltzmann--Gibbs entropy is a functional on the space of probability measures. When a state space is countable, one characterization of the Boltzmann--Gibbs entropy is given by the Shannon--Khinchin axioms, which consist of continuity, maximality, expandability and extensivity. Among these four properties, the extensivity is generalized in various ways. The extensivity of a functional is interpreted as the property that, for any random variables $(X,Y)$ taking finitely many values in $\mathbb{N}$, the difference between the value of the functional at the joint law of $(X,Y)$ and that at the law of $X$ coincides with the linear combinations of the values at the conditional laws of $Y$ given $X=n$ with coefficients given by the probabilities of each event $X=n$. A generalization of the extensivity obtained by replacing the coefficients with a power of the probabilities of the events $X=n$ provides a characterization of the Tsallis entropy.
In this paper, we first prove the impossibility to replace the coefficients with a non-power function of the probabilities of the events $X=n$. Then we estimate the difference between the value at the joint law of $(X,Y)$ and that at the law of $X$ for a general functional.
△ Less
Submitted 16 November, 2020;
originally announced November 2020.
-
Behaviors of $φ$-exponential distributions in Wasserstein geometry and an evolution equation
Authors:
Asuka Takatsu
Abstract:
A $φ$-exponential distribution is a generalization of an exponential distribution associated to functions $φ$ in an appropriate class, and the space of $φ$-exponential distributions has a dually flat structure. We study features of the space of $φ$-exponential distributions, such as the convexity in Wasserstein geometry and the stability under an evolution equation. From this study, we provide the…
▽ More
A $φ$-exponential distribution is a generalization of an exponential distribution associated to functions $φ$ in an appropriate class, and the space of $φ$-exponential distributions has a dually flat structure. We study features of the space of $φ$-exponential distributions, such as the convexity in Wasserstein geometry and the stability under an evolution equation. From this study, we provide the new characterizations to the space of Gaussian measures and the space of $q$-Gaussian measures.
△ Less
Submitted 30 September, 2011;
originally announced September 2011.
