-
Convergence analysis of a primal-dual optimization-by-continuation algorithm
Abstract: We present a numerical iterative optimization algorithm for the minimization of a cost function consisting of a linear combination of three convex terms, one of which is differentiable, a second one is prox-simple and the third one is the composition of a linear map and a prox-simple function. The algorithm's special feature lies in its ability to approximate, in a single iteration run, the minimi… ▽ More
Submitted 15 November, 2023; originally announced November 2023.
Comments: 22 pages, 2 figures
MSC Class: 90C06; 90C25; 90C59; 90C90; 49M29; 65K10
-
On a fixed-point continuation method for a convex optimization problem
Abstract: We consider a variation of the classical proximal-gradient algorithm for the iterative minimization of a cost function consisting of a sum of two terms, one smooth and the other prox-simple, and whose relative weight is determined by a penalty parameter. This so-called fixed-point continuation method allows one to approximate the problem's trade-off curve, i.e. to compute the minimizers of the cos… ▽ More
Submitted 23 December, 2022; originally announced December 2022.
Comments: 15 pages, 2 figures. Workshop on Advanced Techniques in Optimization for Machine learning and Imaging
-
Primal-dual splitting scheme with backtracking for handling with epigraphic constraint and sparse analysis regularization
Abstract: The convergence of many proximal algorithms involving a gradient descent relies on its Lipschitz constant. To avoid computing it, backtracking rules can be used. While such a rule has already been designed for the forward-backward algorithm (FBwB), this scheme is not flexible enough when a non-differentiable penalization with a linear operator is added to a constraint. In this work we propose a ba… ▽ More
Submitted 8 September, 2020; originally announced September 2020.
Comments: in Proceedings of iTWIST'20, Paper-ID: 12, Nantes, France, December, 2-4, 2020
-
arXiv:1806.07677 [pdf, ps, other]
On starting and stopping criteria for nested primal-dual iterations
Abstract: The importance of an adequate inner loop starting point (as opposed to a sufficient inner loop stopping rule) is discussed in the context of a numerical optimization algorithm consisting of nested primal-dual proximal-gradient iterations. While the number of inner iterations is fixed in advance, convergence of the whole algorithm is still guaranteed by virtue of a warm-start strategy for the inner… ▽ More
Submitted 20 June, 2018; originally announced June 2018.
Comments: 18 pages, no figures
MSC Class: 65K10 90C06 90C25 90C90
-
arXiv:1605.03791 [pdf, ps, other]
On the convergence of a linesearch based proximal-gradient method for nonconvex optimization
Abstract: We consider a variable metric linesearch based proximal gradient method for the minimization of the sum of a smooth, possibly nonconvex function plus a convex, possibly nonsmooth term. We prove convergence of this iterative algorithm to a critical point if the objective function satisfies the Kurdyka-Lojasiewicz property at each point of its domain, under the assumption that a limit point exists.… ▽ More
Submitted 8 April, 2017; v1 submitted 12 May, 2016; originally announced May 2016.
MSC Class: 65K05; 90C30; 68U10
Journal ref: Inverse Problems 33 (2017), 055005
-
arXiv:1506.00385 [pdf, ps, other]
Variable metric inexact line-search based methods for nonsmooth optimization
Abstract: We develop a new proximal-gradient method for minimizing the sum of a differentiable, possibly nonconvex, function plus a convex, possibly non differentiable, function. The key features of the proposed method are the definition of a suitable descent direction, based on the proximal operator associated to the convex part of the objective function, and an Armijo-like rule to determine the step size… ▽ More
Submitted 1 June, 2015; originally announced June 2015.
MSC Class: 65K05; 90C30
Journal ref: SIAM Journal on Optimization 26 (2016), 891-921
-
Proceedings of the second "international Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST'14)
Abstract: The implicit objective of the biennial "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For its second edition, the iTWIST workshop took place in the medieval and picturesque town of Namur in… ▽ More
Submitted 9 October, 2014; v1 submitted 2 October, 2014; originally announced October 2014.
Comments: 69 pages, 24 extended abstracts, iTWIST'14 website: http://sites.google.com/site/itwist14
-
arXiv:1302.6088 [pdf, ps, other]
An efficient algorithm for structured sparse quantile regression
Abstract: Quantile regression is studied in combination with a penalty which promotes structured (or group) sparsity. A mixed $\ell_{1,\infty}$-norm on the parameter vector is used to impose structured sparsity on the traditional quantile regression problem. An algorithm is derived to calculate the piece-wise linear solution path of the corresponding minimization problem. A Matlab implementation of the prop… ▽ More
Submitted 25 February, 2013; originally announced February 2013.
Comments: 15 pages, 4 figures
-
arXiv:1203.4451 [pdf, ps, other]
Iterative algorithms for total variation-like reconstructions in seismic tomography
Abstract: A qualitative comparison of total variation like penalties (total variation, Huber variant of total variation, total generalized variation, ...) is made in the context of global seismic tomography. Both penalized and constrained formulations of seismic recovery problems are treated. A number of simple iterative recovery algorithms applicable to these problems are described. The convergence speed o… ▽ More
Submitted 6 April, 2012; v1 submitted 20 March, 2012; originally announced March 2012.
Comments: 28 pages, 8 figures. Corrected sign errors in formula (25)
-
arXiv:1202.3362 [pdf, ps, other]
An iterative algorithm for sparse and constrained recovery with applications to divergence-free current reconstructions in magneto-encephalography
Abstract: We propose an iterative algorithm for the minimization of a $\ell_1$-norm penalized least squares functional, under additional linear constraints. The algorithm is fully explicit: it uses only matrix multiplications with the three matrices present in the problem (in the linear constraint, in the data misfit part and in penalty term of the functional). None of the three matrices must be invertible.… ▽ More
Submitted 15 February, 2012; originally announced February 2012.
Comments: 21 pages, 3 figures
-
Wavelets and wavelet-like transforms on the sphere and their application to geophysical data inversion
Abstract: Many flexible parameterizations exist to represent data on the sphere. In addition to the venerable spherical harmonics, we have the Slepian basis, harmonic splines, wavelets and wavelet-like Slepian frames. In this paper we focus on the latter two: spherical wavelets developed for geophysical applications on the cubed sphere, and the Slepian "tree", a new construction that combines a quadratic co… ▽ More
Submitted 8 September, 2011; originally announced September 2011.
Comments: 15 pages, 11 figures, submitted to the Proceedings of the SPIE 2011 conference Wavelets and Sparsity XIV
-
Solving or resolving global tomographic models with spherical wavelets, and the scale and sparsity of seismic heterogeneity
Abstract: We propose a class of spherical wavelet bases for the analysis of geophysical models and forthe tomographic inversion of global seismic data. Its multiresolution character allows for modeling with an effective spatial resolution that varies with position within the Earth. Our procedure is numerically efficient and can be implemented with parallel computing. We discuss two possible types of discret… ▽ More
Submitted 14 April, 2011; originally announced April 2011.
Comments: 43 pages, 11 figures, submitted to Geophysical Journal International
-
arXiv:1104.1087 [pdf, ps, other]
On a generalization of the iterative soft-thresholding algorithm for the case of non-separable penalty
Abstract: An explicit algorithm for the minimization of an $\ell_1$ penalized least squares functional, with non-separable $\ell_1$ term, is proposed. Each step in the iterative algorithm requires four matrix vector multiplications and a single simple projection on a convex set (or equivalently thresholding). Convergence is proven and a 1/N convergence rate is derived for the functional. In the special case… ▽ More
Submitted 4 August, 2011; v1 submitted 6 April, 2011; originally announced April 2011.
Comments: 17 pages; 1 figure; results formulated for a more general penalty than previous version; numerical example added
Journal ref: Inverse Problems 27 (2011) 125007
-
arXiv:0908.3636 [pdf, ps, other]
Practical error estimates for sparse recovery in linear inverse problems
Abstract: The effectiveness of using model sparsity as a priori information when solving linear inverse problems is studied. We investigate the reconstruction quality of such a method in the non-idealized case and compute some typical recovery errors (depending on the sparsity of the desired solution, the number of data, the noise level on the data, and various properties of the measurement matrix); they ar… ▽ More
Submitted 16 July, 2010; v1 submitted 25 August, 2009; originally announced August 2009.
Comments: 11 pages, 5 figures
MSC Class: 15A29; 65F22; 65F35
-
arXiv:0902.4424 [pdf, ps, other]
Accelerating gradient projection methods for $\ell_1$-constrained signal recovery by steplength selection rules
Abstract: We propose a new gradient projection algorithm that compares favorably with the fastest algorithms available to date for $\ell_1$-constrained sparse recovery from noisy data, both in the compressed sensing and inverse problem frameworks. The method exploits a line-search along the feasible direction and an adaptive steplength selection based on recent strategies for the alternation of the well-k… ▽ More
Submitted 25 February, 2009; originally announced February 2009.
Comments: 11 pages, 4 figures
-
arXiv:0808.3472 [pdf, ps, other]
Nonlinear regularization techniques for seismic tomography
Abstract: The effects of several nonlinear regularization techniques are discussed in the framework of 3D seismic tomography. Traditional, linear, $\ell_2$ penalties are compared to so-called sparsity promoting $\ell_1$ and $\ell_0$ penalties, and a total variation penalty. Which of these algorithms is judged optimal depends on the specific requirements of the scientific experiment. If the correct reproduct… ▽ More
Submitted 18 August, 2010; v1 submitted 26 August, 2008; originally announced August 2008.
Comments: 23 pages, 7 figures. Typographical error corrected in accelerated algorithms (14) and (20)
-
arXiv:0710.4082 [pdf, ps, other]
On the performance of algorithms for the minimization of $\ell_1$-penalized functionals
Abstract: The problem of assessing the performance of algorithms used for the minimization of an $\ell_1$-penalized least-squares functional, for a range of penalty parameters, is investigated. A criterion that uses the idea of `approximation isochrones' is introduced. Five different iterative minimization algorithms are tested and compared, as well as two warm-start strategies. Both well-conditioned and… ▽ More
Submitted 8 January, 2009; v1 submitted 22 October, 2007; originally announced October 2007.
Comments: 18 pages, 10 figures; v3: expanded version with an additional synthetic test problems
-
arXiv:0710.3728 [pdf, ps, other]
L1Packv2: A Mathematica package for minimizing an $\ell_1$-penalized functional
Abstract: L1Packv2 is a Mathematica package that contains a number of algorithms that can be used for the minimization of an $\ell_1$-penalized least squares functional. The algorithms can handle a mix of penalized and unpenalized variables. Several instructive examples are given. Also, an implementation that yields an exact output whenever exact data are given is provided.
Submitted 20 August, 2008; v1 submitted 19 October, 2007; originally announced October 2007.
Comments: 17 pages, 3 figures; v3: Major re-arragangement/reworking of content. Replaced some examples. v4: minor typos and additions
-
arXiv:0708.0046 [pdf, ps, other]
Sparse and stable Markowitz portfolios
Abstract: We consider the problem of portfolio selection within the classical Markowitz mean-variance framework, reformulated as a constrained least-squares regression problem. We propose to add to the objective function a penalty proportional to the sum of the absolute values of the portfolio weights. This penalty regularizes (stabilizes) the optimization problem, encourages sparse portfolios (i.e. portf… ▽ More
Submitted 29 May, 2008; v1 submitted 31 July, 2007; originally announced August 2007.
Comments: Better emphasis of main result, new abstract, new examples and figures. New appendix with full details of algorithm. 17 pages, 6 figures
MSC Class: 62P20; 91B28
-
arXiv:0706.4297 [pdf, ps, other]
Accelerated Projected Gradient Method for Linear Inverse Problems with Sparsity Constraints
Abstract: Regularization of ill-posed linear inverse problems via $\ell_1$ penalization has been proposed for cases where the solution is known to be (almost) sparse. One way to obtain the minimizer of such an $\ell_1$ penalized functional is via an iterative soft-thresholding algorithm. We propose an alternative implementation to $\ell_1$-constraints, using a gradient method, with projection on $\ell_1$-… ▽ More
Submitted 27 August, 2008; v1 submitted 28 June, 2007; originally announced June 2007.
Comments: 24 pages, 5 figures. v2: added reference, some amendments, 27 pages
-
Tomographic inversion using $\ell_1$-norm regularization of wavelet coefficients
Abstract: We propose the use of $\ell_1$ regularization in a wavelet basis for the solution of linearized seismic tomography problems $Am=d$, allowing for the possibility of sharp discontinuities superimposed on a smoothly varying background. An iterative method is used to find a sparse solution $m$ that contains no more fine-scale structure than is necessary to fit the data $d$ to within its assigned err… ▽ More
Submitted 8 August, 2006; originally announced August 2006.
Comments: 19 pages, 14 figures. Submitted to GJI July 2006. This preprint does not use GJI style files (which gives wrong received/accepted dates). Corrected typo
Journal ref: Geophysical Journal International 170 (2007) 359-370
-
Affine Toda-Sutherland Systems
Abstract: A cross between two well-known integrable multi-particle dynamics, an affine Toda molecule and a Sutherland system, is introduced for any affine root system. Though it is not completely integrable but partially integrable, or quasi exactly solvable, it inherits many remarkable properties from the parents. The equilibrium position is algebraic, i.e. proportional to the Weyl vector. The frequencie… ▽ More
Submitted 7 September, 2003; originally announced September 2003.
Comments: LaTeX2e 22 pages with amsfonts and graphicx, 5 eps figures
Report number: YITP-03-64, IP/BBSR/03-13
Journal ref: J.Phys.A37:1665-1680,2004
-
Classical vs Quantum Mechanics: role of elementary excitations
Abstract: Simple theorems relating a quantum mechanical system to the corresponding classical one at equilibrium and connecting the quantum eigenvalues to the frequencies of normal modes oscillations are presented. Corresponding to each quantum eigenfunction, a ` classical eigenfunction' is associated. Those belonging to `elementary excitations' play an important role.
Submitted 16 October, 2003; v1 submitted 7 August, 2003; originally announced August 2003.
Comments: LaTeX2e with amsfonts, 9 pages, no figures. Typos corrected. Three references added. Section 3 with summary and comments added
Report number: YITP-03-50
Journal ref: Physics Letters A327 (2004) 152--157
-
Quantum & Classical Eigenfunctions in Calogero & Sutherland Systems
Abstract: An interesting observation was reported by Corrigan-Sasaki that all the frequencies of small oscillations around equilibrium are " quantised" for Calogero and Sutherland (C-S) systems, typical integrable multi-particle dynamics. We present an analytic proof by applying recent results of Loris-Sasaki. Explicit forms of `classical' and quantum eigenfunctions are presented for C-S systems based on… ▽ More
Submitted 9 September, 2003; v1 submitted 7 August, 2003; originally announced August 2003.
Comments: LaTeX2e 37 pages, references added, typo corrected, a few paragraphs added
Report number: YITP-03-51
Journal ref: J.Phys.A37:211-238,2004