Optimization and Control
See recent articles
- [1] arXiv:2408.16918 [pdf, html, other]
-
Title: Non-Monotone Variational InequalitiesSubjects: Optimization and Control (math.OC)
In this paper, we focus on deriving some sufficient conditions for the existence of solutions to non-monotone Variational Inequalities (VIs) based on inverse mapping theory. We have obtained several widely applicable sufficient conditions for this problem and have introduced a sufficient condition for the existence of a Minty solution. We have shown that the extra-gradient method converges to a solution of VI in the presence of a Minty solution. Additionally, we have shown that, under some extra assumption, the algorithm is efficient and approaches a particular type of Minty solution. Interpreting these results in an equivalent game theory problem, weak coupling conditions will be obtained, stating that if the players' cost functions are sufficiently weakly coupled, the game has a pure quasi-Nash equilibrium. Moreover, under the additional assumption of the existence of Minty solutions, a pure Nash equilibrium exists for the corresponding game.
- [2] arXiv:2408.16927 [pdf, html, other]
-
Title: Serial and Parallel Two-Column Probing for Mixed-Integer ProgrammingComments: 22 pages, 2 figures, 3 chartsSubjects: Optimization and Control (math.OC)
Probing in mixed-integer programming (MIP) is a technique of temporarily fixing variables to discover implications that are useful to branch-and-cut solvers. Such fixing is typically performed one variable at a time -- this paper develops instead a two-column probing scheme that instead fixes a pair of variables per iteration. Although the scheme involves more work per iteration compared to the one-column approach, stronger implied bounds as well as more conflicts identified may compensate. Indeed, our prototype implementation was awarded first prize at the MIP Workshop 2024 Computational Competition on novel presolving approaches. This paper presents the aforementioned (serial) prototype and additionally develops an efficient parallelization, leveraging hardware acceleration to further improve overall solve times. Compared to serial two-column probing, our parallel version sacrifices some strength per-pair probed in exchange for greatly increasing the total number of such probings; computational experiments demonstrate its promise.
- [3] arXiv:2408.16989 [pdf, other]
-
Title: Optimal Ratcheting of Dividends with Irreversible ReinsuranceComments: 62 pages, 1 figureSubjects: Optimization and Control (math.OC); Probability (math.PR); Pricing of Securities (q-fin.PR); Risk Management (q-fin.RM)
This paper considers an insurance company that faces two key constraints: a ratcheting dividend constraint and an irreversible reinsurance constraint. The company allocates part of its reserve to pay dividends to its shareholders while strategically purchasing reinsurance for its claims. The ratcheting dividend constraint ensures that dividend cuts are prohibited at any time. The irreversible reinsurance constraint ensures that reinsurance contracts cannot be prematurely terminated or sold to external entities. The dividend rate level and the reinsurance level are modelled as nondecreasing processes, thereby satisfying the constraints. The incurred claims are modelled via a Brownian risk model. The main objective is to maximize the cumulative expected discounted dividend payouts until the time of ruin. The reinsurance and dividend levels belong to either a finite set or a closed interval. The optimal value functions for the finite set case and the closed interval case are proved to be the unique viscosity solutions of the corresponding Hamilton-Jacobi-Bellman equations, and the convergence between these optimal value functions is established. For the finite set case, a threshold strategy is proved to be optimal, while for the closed interval case, an $\epsilon$-optimal strategy is constructed. Finally, numerical examples are presented to illustrate the optimality conditions and optimal strategies.
- [4] arXiv:2408.17030 [pdf, other]
-
Title: Zero-sum stochastic linear-quadratic Stackelberg differential games of Markovian regime-switching systemSubjects: Optimization and Control (math.OC)
This paper investigates a zero-sum stochastic linear-quadratic (SLQ, for short) Stackelberg differential game problem, where the coefficients of the state equation and the weighting matrices in the performance functional are regulated by a Markov chain. By utilizing the findings in \citet{Zhang.X.2021_ILQM}, we directly present the feedback representation to the rational reaction of the follower. For the leader's problem, we derive the optimality system through the variational method and study its unique solvability from the Hilbert space point of view. We construct the explicit optimal control for the leader based on the solution to coupled differential Riccati equations (CDREs, for short) and obtain the solvability of CDREs under the one-dimensional framework. Finally, we provide two concrete examples to illustrate the results developed in this paper.
- [5] arXiv:2408.17113 [pdf, other]
-
Title: A Two-Timescale Decision-Hazard-Decision Formulation for Storage Usage Values CalculationCamila Martinez Parra (RTE, CERMICS), Michel de Lara (CERMICS), Jean-Philippe Chancelier (CERMICS), Pierre Carpentier (UMA), Jean-Marc Janin (RTE), Manuel Ruiz (RTE)Subjects: Optimization and Control (math.OC)
The penetration of renewable energies requires additional storages to deal with intermittency. Accordingly, there is growing interest in evaluating the opportunity cost (usage value) associated with stored energy in large storages, a cost obtained by solving a multistage stochastic optimization problem. Today, to compute usage values under uncertainties, an adequacy resource problem is solved using stochastic dynamic programming assuming a hazard-decision information structure. This modelling assumes complete knowledge of the coming week uncertainties, which is not adapted to the system operation as the intermittency occurs at smaller timescale. We equip the twotimescale problem with a new information structure considering planning and recourse decisions: decision-hazard-decision. This structure is used to decompose the multistage decision-making process into a nonanticipative planning step in which the on/off decisions for the thermal units are made, and a recourse step in which the power modulation decisions are made once the uncertainties have been disclosed. In a numerical case, we illustrate how usage values are sensitive as how the disclosure of information is modelled.
- [6] arXiv:2408.17156 [pdf, html, other]
-
Title: Asynchronous Distributed Learning with Quantized Finite-Time CoordinationComments: To be presented at 2024 IEEE Conference on Decision and ControlSubjects: Optimization and Control (math.OC); Systems and Control (eess.SY)
In this paper we address distributed learning problems over peer-to-peer networks. In particular, we focus on the challenges of quantized communications, asynchrony, and stochastic gradients that arise in this set-up. We first discuss how to turn the presence of quantized communications into an advantage, by resorting to a finite-time, quantized coordination scheme. This scheme is combined with a distributed gradient descent method to derive the proposed algorithm. Secondly, we show how this algorithm can be adapted to allow asynchronous operations of the agents, as well as the use of stochastic gradients. Finally, we propose a variant of the algorithm which employs zooming-in quantization. We analyze the convergence of the proposed methods and compare them to state-of-the-art alternatives.
- [7] arXiv:2408.17173 [pdf, html, other]
-
Title: Existence and approximate controllability results for time-fractional stochastic Navier-Stokes equationsSubjects: Optimization and Control (math.OC); Dynamical Systems (math.DS)
This paper deals with time-fractional stochastic Navier-Stokes equations, which are characterized by the coexistence of stochastic noise and a fractional power of the Laplacian. We establish sufficient conditions for the existence and approximate controllability of a unique mild solution to time-fractional stochastic Navier-Stokes equations. Using a fixed point technique, we first demonstrate the existence and uniqueness of a mild solution to the equation under consideration. We then establish approximate controllability results by using the concepts of fractional calculus, semigroup theory, functional analysis and stochastic analysis.
- [8] arXiv:2408.17196 [pdf, html, other]
-
Title: An Equilibrium Dynamic Traffic Assignment Model with Linear Programming FormulationSubjects: Optimization and Control (math.OC); Computer Science and Game Theory (cs.GT)
In this paper, we consider a dynamic equilibrium transportation problem. There is a fixed number of cars moving from origin to destination areas. Preferences for arrival times are expressed as a cost of arriving before or after the preferred time at the destination. Each driver aims to minimize the time spent during the trip, making the time spent a measure of cost. The chosen routes and departure times impact the network loading. The goal is to find an equilibrium distribution across departure times and routes.
For a relatively simplified transportation model we show that an equilibrium traffic distribution can be found as a solution to a linear program. In earlier works linear programming formulations were only obtained for social optimum dynamic traffic assignment problems. We also discuss algorithmic approaches for solving the equilibrium problem using time-expanded networks. - [9] arXiv:2408.17213 [pdf, html, other]
-
Title: A Minimization Approach for Minimax Optimization with Coupled ConstraintsComments: 25 pagesSubjects: Optimization and Control (math.OC)
In this paper, we focus on the nonconvex-strongly-concave minimax optimization problem (MCC), where the inner maximization subproblem contains constraints that couple the primal variable of the outer minimization problem. We prove that by introducing the dual variable of the inner maximization subproblem, (MCC) has the same first-order minimax points as a nonconvex-strongly-concave minimax optimization problem without coupled constraints (MOL). We then extend our focus to a class of nonconvex-strongly-concave minimax optimization problems (MM) that generalize (MOL). By performing the partial forward-backward envelope to the primal variable of the inner maximization subproblem, we propose a minimization problem (MMPen), where its objective function is explicitly formulated. We prove that the first-order stationary points of (MMPen) coincide with the first-order minimax points of (MM). Therefore, various efficient minimization methods and their convergence guarantees can be directly employed to solve (MM), hence solving (MCC) through (MOL). Preliminary numerical experiments demonstrate the great potential of our proposed approach.
- [10] arXiv:2408.17246 [pdf, html, other]
-
Title: Learning and Verifying Maximal Taylor-Neural Lyapunov functionsSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Systems and Control (eess.SY)
We introduce a novel neural network architecture, termed Taylor-neural Lyapunov functions, designed to approximate Lyapunov functions with formal certification. This architecture innovatively encodes local approximations and extends them globally by leveraging neural networks to approximate the residuals. Our method recasts the problem of estimating the largest region of attraction - specifically for maximal Lyapunov functions - into a learning problem, ensuring convergence around the origin through robust control theory. Physics-informed machine learning techniques further refine the estimation of the largest region of attraction. Remarkably, this method is versatile, operating effectively even without simulated data points. We validate the efficacy of our approach by providing numerical certificates of convergence across multiple examples. Our proposed methodology not only competes closely with state-of-the-art approaches, such as sum-of-squares and LyZNet, but also achieves comparable results even in the absence of simulated data. This work represents a significant advancement in control theory, with broad potential applications in the design of stable control systems and beyond.
- [11] arXiv:2408.17288 [pdf, html, other]
-
Title: An Integer Linear Programming Model for Earth Observation MissionsSubjects: Optimization and Control (math.OC); Numerical Analysis (math.NA)
This paper addresses an optimization problem in satellite observation mission planning, focusing on the challenges of decentralized decision-making among satellites, which is crucial for optimizing strategies in dynamic observation environments. The method integrates mathematical modeling using integer programming and time-varying communication graphs, which are essential for efficient task scheduling. Specifically, the approach utilizes distributed Lagrangian relaxation techniques to manage the complexity of the problem. Numerical simulations are conducted to explore the feasibility of the proposed approach for handling complex satellite operations under evolving communication dynamics.
- [12] arXiv:2408.17295 [pdf, html, other]
-
Title: All You Need is Group Actions: Advancing Robust Autonomous PlanningSubjects: Optimization and Control (math.OC); Group Theory (math.GR); Numerical Analysis (math.NA)
Managing the plan of constellation of satellites for target observation requires optimal deployment and efficient operational strategies. In this paper, we introduce a new technique based on group theory tools through multi-agent constraint optimization techniques, designed for the dynamic landscapes of satellite operations. Inspired by group actions, our method models the planning problem for observing Earth targets as a cooperative game to achieve computational efficiency while simultaneously reducing computational complexity. Designed for the complex task of planning constellation of satellites, our methodology provides a feasible solution to the inherent challenges of multi-agent optimization under state constraints and subject to uncertainties. Our approach can offer avenues for improving mission efficiency and reducing costs. Through numerical simulations, we demonstrate the good performance of the approach in the presence of inter-satellite links.
- [13] arXiv:2408.17342 [pdf, html, other]
-
Title: Optimal Control Problems with Vector-Valued Impulse Controls and Time DelaysSubjects: Optimization and Control (math.OC)
We consider a nonlinear control system with vector-valued measures as controls and with dynamics depending on time delayed states. First, we introduce a notion of discontinuous, bounded variation solution associated with this system and establish an equivalent representation formula for it, inspired by the approach known in delay-free impulsive control as the `graph completions' method. Then, thanks to this equivalent formulation, we prove well-posedness properties of these solutions and also derive necessary optimality conditions in the form of a Maximum Principle for an associated minimization problem.
- [14] arXiv:2408.17350 [pdf, html, other]
-
Title: Regular Pairings for Non-quadratic Lyapunov Functions and Contraction AnalysisSubjects: Optimization and Control (math.OC); Systems and Control (eess.SY); Dynamical Systems (math.DS); Functional Analysis (math.FA)
Recent studies on stability and contractivity have highlighted the importance of semi-inner products, which we refer to as ``pairings'', associated with general norms. A pairing is a binary operation that relates the derivative of a curve's norm to the radius-vector of the curve and its tangent. This relationship, known as the curve norm derivative formula, is crucial when using the norm as a Lyapunov function. Another important property of the pairing, used in stability and contraction criteria, is the so-called Lumer inequality, which relates the pairing to the induced logarithmic norm. We prove that the curve norm derivative formula and Lumer's inequality are, in fact, equivalent to each other and to several simpler properties. We then introduce and characterize regular pairings that satisfy all of these properties. Our results unify several independent theories of pairings (semi-inner products) developed in previous work on functional analysis and control theory. Additionally, we introduce the polyhedral max pairing and develop computational tools for polyhedral norms, advancing contraction theory in non-Euclidean spaces.
New submissions for Monday, 2 September 2024 (showing 14 of 14 entries )
- [15] arXiv:2408.16852 (cross-list from cs.LG) [pdf, html, other]
-
Title: The Star Geometry of Critic-Based Regularizer LearningSubjects: Machine Learning (cs.LG); Metric Geometry (math.MG); Optimization and Control (math.OC); Statistics Theory (math.ST); Machine Learning (stat.ML)
Variational regularization is a classical technique to solve statistical inference tasks and inverse problems, with modern data-driven approaches parameterizing regularizers via deep neural networks showcasing impressive empirical performance. Recent works along these lines learn task-dependent regularizers. This is done by integrating information about the measurements and ground-truth data in an unsupervised, critic-based loss function, where the regularizer attributes low values to likely data and high values to unlikely data. However, there is little theory about the structure of regularizers learned via this process and how it relates to the two data distributions. To make progress on this challenge, we initiate a study of optimizing critic-based loss functions to learn regularizers over a particular family of regularizers: gauges (or Minkowski functionals) of star-shaped bodies. This family contains regularizers that are commonly employed in practice and shares properties with regularizers parameterized by deep neural networks. We specifically investigate critic-based losses derived from variational representations of statistical distances between probability measures. By leveraging tools from star geometry and dual Brunn-Minkowski theory, we illustrate how these losses can be interpreted as dual mixed volumes that depend on the data distribution. This allows us to derive exact expressions for the optimal regularizer in certain cases. Finally, we identify which neural network architectures give rise to such star body gauges and when do such regularizers have favorable properties for optimization. More broadly, this work highlights how the tools of star geometry can aid in understanding the geometry of unsupervised regularizer learning.
- [16] arXiv:2408.16890 (cross-list from cs.RO) [pdf, html, other]
-
Title: Robotic warehousing operations: a learn-then-optimize approach to large-scale neighborhood searchSubjects: Robotics (cs.RO); Machine Learning (cs.LG); Optimization and Control (math.OC)
The rapid deployment of robotics technologies requires dedicated optimization algorithms to manage large fleets of autonomous agents. This paper supports robotic parts-to-picker operations in warehousing by optimizing order-workstation assignments, item-pod assignments and the schedule of order fulfillment at workstations. The model maximizes throughput, while managing human workload at the workstations and congestion in the facility. We solve it via large-scale neighborhood search, with a novel learn-then-optimize approach to subproblem generation. The algorithm relies on an offline machine learning procedure to predict objective improvements based on subproblem features, and an online optimization model to generate a new subproblem at each iteration. In collaboration with Amazon Robotics, we show that our model and algorithm generate much stronger solutions for practical problems than state-of-the-art approaches. In particular, our solution enhances the utilization of robotic fleets by coordinating robotic tasks for human operators to pick multiple items at once, and by coordinating robotic routes to avoid congestion in the facility.
- [17] arXiv:2408.16899 (cross-list from eess.SY) [pdf, html, other]
-
Title: Mitigating Polarization in Recommender Systems via Network-aware Feedback OptimizationSubjects: Systems and Control (eess.SY); Optimization and Control (math.OC)
We consider a recommender system that takes into account the interaction between recommendations and the evolution of user interests. Users opinions are influenced by both social interactions and recommended content. We leverage online feedback optimization to design a recommender system that trades-off between maximizing engagement and minimizing polarization. The recommender system is agnostic about users' opinion, clicking behavior, and social interactions, and solely relies on clicks. We establish optimality and closed-loop stability of the resulting feedback interconnection between the social platform and the recommender system. We numerically validate our algorithm when the user population follows an extended Friedkin--Johnsen model. We observe that network-aware recommendations significantly reduce polarization without compromising user engagement.
- [18] arXiv:2408.16981 (cross-list from cs.LG) [pdf, html, other]
-
Title: The Sample-Communication Complexity Trade-off in Federated Q-LearningSubjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
We consider the problem of federated Q-learning, where $M$ agents aim to collaboratively learn the optimal Q-function of an unknown infinite-horizon Markov decision process with finite state and action spaces. We investigate the trade-off between sample and communication complexities for the widely used class of intermittent communication algorithms. We first establish the converse result, where it is shown that a federated Q-learning algorithm that offers any speedup with respect to the number of agents in the per-agent sample complexity needs to incur a communication cost of at least an order of $\frac{1}{1-\gamma}$ up to logarithmic factors, where $\gamma$ is the discount factor. We also propose a new algorithm, called Fed-DVR-Q, which is the first federated Q-learning algorithm to simultaneously achieve order-optimal sample and communication complexities. Thus, together these results provide a complete characterization of the sample-communication complexity trade-off in federated Q-learning.
- [19] arXiv:2408.17109 (cross-list from math.PR) [pdf, other]
-
Title: Sensitivity of causal distributionally robust optimizationComments: 33 pagesSubjects: Probability (math.PR); Optimization and Control (math.OC)
We study the causal distributionally robust optimization (DRO) in both discrete- and continuous- time settings. The framework captures model uncertainty, with potential models penalized in function of their adapted Wasserstein distance to a given reference model. Strength of the penalty is controlled using a real-valued parameter, which is simply the radius of the uncertainty ball in the special case of an indicator penalty. Our main results derive the first-order sensitivity of the value of causal DRO with respect to the penalization parameter, i.e., we compute the sensitivity to model uncertainty. Moreover, we investigate the case where a martingale constraint is imposed on the underlying model, as is the case for pricing measures in mathematical finance. We introduce different scaling regimes, which allow us to obtain the continuous-time sensitivities as nontrivial limits of their discrete-time counterparts. We illustrate our results with examples.
Our proofs rely on novel methods. In particular, we introduce pathwise Malliavin derivative, which agree a.s. with its classical counterpart under the Wiener measure, and we extend the adjoint operator, the Skorokhod integral, to regular martingale integrators and show it satisfies a stochastic Fubini theorem. - [20] arXiv:2408.17145 (cross-list from cs.LG) [pdf, html, other]
-
Title: Towards Hyper-parameter-free Federated LearningComments: 28 pages, 3 figuresSubjects: Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Optimization and Control (math.OC)
The adaptive synchronization techniques in federated learning (FL) for scaled global model updates show superior performance over the vanilla federated averaging (FedAvg) scheme. However, existing methods employ additional tunable hyperparameters on the server to determine the scaling factor. A contrasting approach is automated scaling analogous to tuning-free step-size schemes in stochastic gradient descent (SGD) methods, which offer competitive convergence rates and exhibit good empirical performance. In this work, we introduce two algorithms for automated scaling of global model updates. In our first algorithm, we establish that a descent-ensuring step-size regime at the clients ensures descent for the server objective. We show that such a scheme enables linear convergence for strongly convex federated objectives. Our second algorithm shows that the average of objective values of sampled clients is a practical and effective substitute for the objective function value at the server required for computing the scaling factor, whose computation is otherwise not permitted. Our extensive empirical results show that the proposed methods perform at par or better than the popular federated learning algorithms for both convex and non-convex problems. Our work takes a step towards designing hyper-parameter-free federated learning.
- [21] arXiv:2408.17387 (cross-list from stat.ML) [pdf, html, other]
-
Title: Bayesian Optimization for Non-Convex Two-Stage Stochastic Optimization ProblemsSubjects: Machine Learning (stat.ML); Machine Learning (cs.LG); Optimization and Control (math.OC)
Bayesian optimization is a sample-efficient method for solving expensive, black-box optimization problems. Stochastic programming concerns optimization under uncertainty where, typically, average performance is the quantity of interest. In the first stage of a two-stage problem, here-and-now decisions must be made in the face of this uncertainty, while in the second stage, wait-and-see decisions are made after the uncertainty has been resolved. Many methods in stochastic programming assume that the objective is cheap to evaluate and linear or convex. In this work, we apply Bayesian optimization to solve non-convex, two-stage stochastic programs which are expensive to evaluate. We formulate a knowledge-gradient-based acquisition function to jointly optimize the first- and second-stage variables, establish a guarantee of asymptotic consistency and provide a computationally efficient approximation. We demonstrate comparable empirical results to an alternative we formulate which alternates its focus between the two variable types, and superior empirical results over the standard, naive, two-step benchmark. We show that differences in the dimension and length scales between the variable types can lead to inefficiencies of the two-step algorithm, while the joint and alternating acquisition functions perform well in all problems tested. Experiments are conducted on both synthetic and real-world examples.
Cross submissions for Monday, 2 September 2024 (showing 7 of 7 entries )
- [22] arXiv:2106.05958 (replaced) [pdf, html, other]
-
Title: High Probability Complexity Bounds for Non-Smooth Stochastic Optimization with Heavy-Tailed NoiseComments: 61 pages, 12 figures. Changes in V2: different presentation of the results, different structure, new experiments. Changes in V3: some typos were fixedSubjects: Optimization and Control (math.OC); Machine Learning (cs.LG)
Stochastic first-order methods are standard for training large-scale machine learning models. Random behavior may cause a particular run of an algorithm to result in a highly suboptimal objective value, whereas theoretical guarantees are usually proved for the expectation of the objective value. Thus, it is essential to theoretically guarantee that algorithms provide small objective residual with high probability. Existing methods for non-smooth stochastic convex optimization have complexity bounds with the dependence on the confidence level that is either negative-power or logarithmic but under an additional assumption of sub-Gaussian (light-tailed) noise distribution that may not hold in practice. In our paper, we resolve this issue and derive the first high-probability convergence results with logarithmic dependence on the confidence level for non-smooth convex stochastic optimization problems with non-sub-Gaussian (heavy-tailed) noise. To derive our results, we propose novel stepsize rules for two stochastic methods with gradient clipping. Moreover, our analysis works for generalized smooth objectives with Hölder-continuous gradients, and for both methods, we provide an extension for strongly convex problems. Finally, our results imply that the first (accelerated) method we consider also has optimal iteration and oracle complexity in all the regimes, and the second one is optimal in the non-smooth setting.
- [23] arXiv:2202.09651 (replaced) [pdf, html, other]
-
Title: An Oracle Gradient Regularized Newton Method for Quadratic Measurements RegressionSubjects: Optimization and Control (math.OC); Statistics Theory (math.ST)
Recovering an unknown signal from quadratic measurements has gained popularity due to its wide range of applications, including phase retrieval, fusion frame phase retrieval, and positive operator-valued measures. In this paper, we employ a least squares approach to reconstruct the signal and establish its non-asymptotic statistical properties. Our analysis shows that the estimator perfectly recovers the true signal in the noiseless case, while the error between the estimator and the true signal is bounded by $O(\sqrt{p\log(1+2n)/n})$ in the noisy case, where $n$ is the number of measurements and $p$ is the dimension of the signal. We then develop a two-phase algorithm, gradient regularized Newton method (GRNM), to solve the least squares problem. It is proven that the first phase terminates within finitely many steps, and the sequence generated in the second phase converges to a unique local minimum at a superlinear rate under certain mild conditions. Beyond these deterministic results, GRNM is capable of exactly reconstructing the true signal in the noiseless case and achieving the stated error rate with a high probability in the noisy case. Numerical experiments demonstrate that GRNM offers a high level of recovery capability and accuracy as well as fast computational speed.
- [24] arXiv:2212.08178 (replaced) [pdf, html, other]
-
Title: Benders Decomposition for Bi-objective Linear ProgramsComments: 42 pages, 15 figuresSubjects: Optimization and Control (math.OC)
In this paper, we develop a new decomposition technique for solving bi-objective linear programming problems. The proposed methodology combines the bi-objective simplex algorithm with Benders decomposition and can be used to obtain a complete set of extreme efficient solutions, and the corresponding set of extreme non-dominated points, for a bi-objective linear program. Using a Benders-like reformulation, the decomposition approach decouples the problem into a bi-objective master problem and a bi-objective subproblem, each of which is solved using the bi-objective parametric simplex algorithm. The master problem provides candidate extreme efficient solutions that the subproblem assesses for feasibility and optimality. As in standard Benders decomposition, optimality and feasibility cuts are generated by the subproblem and guide the master problem solve. This paper discusses bi-objective Benders decomposition from a theoretical perspective, proves the correctness of the proposed reformulation and addresses the need for so-called weighted optimality cuts. Furthermore, we present an algorithm to solve the reformulation and discuss its performance for three types of bi-objective optimisation problems.
- [25] arXiv:2301.04204 (replaced) [pdf, html, other]
-
Title: A Newton-CG based barrier-augmented Lagrangian method for general nonconvex conic optimizationComments: To appear in Computational Optimization and Applications. arXiv admin note: text overlap with arXiv:2301.03139Subjects: Optimization and Control (math.OC); Machine Learning (cs.LG); Numerical Analysis (math.NA); Machine Learning (stat.ML)
In this paper we consider finding an approximate second-order stationary point (SOSP) of general nonconvex conic optimization that minimizes a twice differentiable function subject to nonlinear equality constraints and also a convex conic constraint. In particular, we propose a Newton-conjugate gradient (Newton-CG) based barrier-augmented Lagrangian method for finding an approximate SOSP of this problem. Under some mild assumptions, we show that our method enjoys a total inner iteration complexity of $\widetilde{\cal O}(\epsilon^{-11/2})$ and an operation complexity of $\widetilde{\cal O}(\epsilon^{-11/2}\min\{n,\epsilon^{-5/4}\})$ for finding an $(\epsilon,\sqrt{\epsilon})$-SOSP of general nonconvex conic optimization with high probability. Moreover, under a constraint qualification, these complexity bounds are improved to $\widetilde{\cal O}(\epsilon^{-7/2})$ and $\widetilde{\cal O}(\epsilon^{-7/2}\min\{n,\epsilon^{-3/4}\})$, respectively. To the best of our knowledge, this is the first study on the complexity of finding an approximate SOSP of general nonconvex conic optimization. Preliminary numerical results are presented to demonstrate superiority of the proposed method over first-order methods in terms of solution quality.
- [26] arXiv:2309.09396 (replaced) [pdf, html, other]
-
Title: Optimality Conditions for Interval-Valued Optimization Problems on Riemannian Manifolds Under a Total Order RelationSubjects: Optimization and Control (math.OC); Differential Geometry (math.DG)
This article explores fundamental properties of convex interval-valued functions defined on Riemannian manifolds. The study employs generalized Hukuhara directional differentiability to derive KKT-type optimality conditions for an interval-valued optimization problem on Riemannian manifolds. Based on type of functions involved in optimization problems, we consider the following cases:
1. objective function as well as constraints are real-valued;
2. objective function is interval-valued, and constraints are real-valued;
3. objective function as well as constraints are interval-valued.
The whole theory is justified with the help of examples. The order relation that we use throughout the paper is a total order relation defined on the collection of all closed and bounded intervals in $\mathbb{R}$. - [27] arXiv:2312.13388 (replaced) [pdf, html, other]
-
Title: Multi-period Stochastic Network Design for Combined Natural Gas and Hydrogen DistributionSubjects: Optimization and Control (math.OC)
Hydrogen is produced from water using renewable electricity. Unlike electricity, hydrogen can be stored in large quantities for long periods. This storage ability acts as a green battery, allowing solar and wind energy to be generated and used at different times. As a result, green hydrogen plays a central role in facilitating a climate-neutral economy. However, the logistics for hydrogen are complex. As new hydrogen pipelines are developed for hydrogen, there is a trend toward repurposing the natural gas network for hydrogen, due to its economic and environmental benefits. Yet, a rapid conversion could disrupt the balance of natural gas supply and demand. Furthermore, technical and economic developments surrounding the transition contribute additional complexity, which introduces uncertainty in future supply and demand levels for both commodities. To address these challenges, we introduce a multi-period stochastic network design problem for the transition of a natural gas pipeline network into a green hydrogen pipeline network. We develop a progressive hedging based matheuristic to solve the problem. Results demonstrate our matheuristic is efficient, both in computation time and in solution quality. We show that factoring in uncertainty avoids premature expansion and ensures the development of an adequate pipeline network meeting long-term needs. In a case study in the Northern Netherlands for Hydrogen Energy Applications in Valley Environments for Northern Netherlands initiative, we focus on two key scenarios: local production and importation, exploring their impacts on performance indicators. Our case insights exemplify the solid foundation for strategic decision-making in energy transitions through our approach.
- [28] arXiv:2402.00471 (replaced) [pdf, other]
-
Title: Fairness by design in shared-energy allocation problemsSubjects: Optimization and Control (math.OC)
This paper studies how to aggregate prosumers (or large consumers) and their collective decisions in electricity markets, with a focus on fairness. Fairness is essential for prosumers to participate in aggregation schemes. Some prosumers may not be able to access the energy market directly, even though it would be beneficial for them. Therefore, new companies offer to aggregate them and promise to treat them fairly. This leads to a fair resource allocation problem.
We propose to use acceptability constraints to guarantee that each prosumer gains from the aggregation. Moreover, we aim to distribute the costs and benefits fairly, taking into account the multi-period and uncertain nature of the problem. Rather than using financial mechanisms to adjust for fairness issues, we focus on various objectives and constraints, within decision problems, that achieve fairness by design. We start from a simple single-period and deterministic model, and then generalize it to a dynamic and stochastic setting using, e.g., stochastic dominance constraints. - [29] arXiv:2402.07560 (replaced) [pdf, html, other]
-
Title: Stabilization of control systems associated with a strongly continuous groupSubjects: Optimization and Control (math.OC)
This paper is devoted to the stabilization of a linear control system $y' = A y + B u$ and its suitable non-linear variants where $(A, \cD(A))$ is an infinitesimal generator of a strongly continuous {\it group} in a Hilbert space $\mH$, and $B$ defined in a Hilbert space $\mU$ is an admissible control operator with respect to the semigroup generated by $A$. Let $\lambda \in \mR$ and assume that, for some {\it positive} symmetric, invertible $Q = Q(\lambda) \in \cL(\mH)$, for some {\it non-negative}, symmetric $R = R(\lambda) \in \cL(\mH)$, and for some {\it non-negative}, symmetric $W = W(\lambda) \in \cL(\mU)$, it holds $$ A Q + Q A^* - B W B^* + Q R Q + 2 \lambda Q = 0. $$ We then present a new approach to study the stabilization of such a system and its suitable nonlinear variants. Both the stabilization using dynamic feedback controls and the stabilization using static feedback controls in a weak sense are investigated. To our knowledge, the nonlinear case is out of reach previously when $B$ is unbounded for both types of stabilization.
- [30] arXiv:2404.00764 (replaced) [pdf, html, other]
-
Title: Sparse Recovery: The Square of $\ell_1/\ell_2$ NormsSubjects: Optimization and Control (math.OC)
This paper introduces a nonconvex approach for sparse signal recovery, proposing a novel model termed the $\tau_2$-model, which utilizes the squared $\ell_1/\ell_2$ norms for this purpose. Our model offers an advancement over the $\ell_0$ norm, which is often computationally intractable and less effective in practical scenarios. Grounded in the concept of effective sparsity, our approach robustly measures the number of significant coordinates in a signal, making it a powerful alternative for sparse signal estimation. The $\tau_2$-model is particularly advantageous due to its computational efficiency and practical applicability.
We detail two accompanying algorithms based on Dinkelbach's procedure and a difference of convex functions strategy. The first algorithm views the model as a linear-constrained quadratic programming problem in noiseless scenarios and as a quadratic-constrained quadratic programming problem in noisy scenarios. The second algorithm, capable of handling both noiseless and noisy cases, is based on the alternating direction linearized proximal method of multipliers. We also explore the model's properties, including the existence of solutions under certain conditions, and discuss the convergence properties of the algorithms. Numerical experiments with various sensing matrices validate the effectiveness of our proposed model. - [31] arXiv:2405.12845 (replaced) [pdf, html, other]
-
Title: Quantum computing and the stable set problemSubjects: Optimization and Control (math.OC)
Given an undirected graph, the stable set problem asks to determine the cardinality of the largest subset of pairwise non-adjacent vertices. This value is called the stability number of the graph, and its computation is an NP-hard problem. In this paper, we solve the stable set problem using the D-wave quantum annealer. By formulating the problem as a quadratic unconstrained binary optimization problem with the penalty method, we show its optimal value equals the graph's stability number for specific penalty values. However, D-Wave's quantum annealer is a heuristic, so the solutions may be far from the optimum and may not represent stable sets. To address these, we introduce a post-processing procedure that identifies samples that could lead to improved solutions. Additionally, we propose a partitioning method to handle larger instances that cannot be embedded on D-Wave's quantum processing unit. Finally, we investigate how different penalty parameter values affect the solutions' quality. Extensive computational results show that the post-processing procedure significantly improves the solution quality, while the partitioning method successfully extends our approach to medium-size instances.
- [32] arXiv:2407.10513 (replaced) [pdf, html, other]
-
Title: On sets of matrices having different spectrum maximizing products with unequal numbers of the same name factorsComments: Changed title, added 2 bibliography referencesSubjects: Optimization and Control (math.OC); Dynamical Systems (math.DS)
Recently, J. Bochi and P. Laskawiec constructed an example of a set of matrices $\{A,B\}$ having two different (up to cyclic permutations of factors) spectrum maximizing products, $AABABB$ and $BBABAA$. In this paper, we identify a class of matrix sets for which the existence of at least one spectrum maximizing product with an odd number of factors automatically entails the existence of another spectrum maximizing product. Moreover, in addition to Bochi--Laskawiec's example, the number of factors of the same name (factors of the form $A$ or $B$) in these matrix products turns out to be different. The efficiency of the proposed approach is confirmed by constructing an example of a set of $2\times2$ matrices $\{A,B\}$ that has spectrum maximizing products of the form $BAA$ and $BBA$.
- [33] arXiv:2408.14412 (replaced) [pdf, html, other]
-
Title: Expanding the Class of Quadratic Control-Lyapunov Functions for Low-Thrust Trajectory OptimizationSubjects: Optimization and Control (math.OC)
Control laws derived from Control-Lyapunov Functions (CLFs) offer an efficient way for generating near-optimal many-revolution low-thrust trajectories. A common approach to constructing CLFs is to consider the family of quadratic functions using a diagonal weighting matrix. In this paper, we explore the advantages of using a larger family of quadratic functions. More specifically, we consider positive-definite weighting matrices with non-zero off-diagonal elements (hereafter referred to as "full" matrices). We propose a novel eigendecomposition method for parameterizing $N$-dimensional weighting matrices that is easy to implement and guarantees positive-definiteness of the weighting matrices. We use particle swarm optimization, which is a stochastic optimization algorithm, to optimize the parameters and generate near-optimal minimum-time low-thrust trajectories. Solutions obtained using a full positive-definite matrix are compared to the results from the (standard) diagonal weighting matrix for a number of benchmark problems. Results demonstrate that improvements in optimality are achieved, especially for maneuvers with large changes in orbital elements.
- [34] arXiv:2009.07799 (replaced) [pdf, html, other]
-
Title: On the Curse of Memory in Recurrent Neural Networks: Approximation and Optimization AnalysisComments: Updated to include the condition $\sup_n \| \boldsymbol{x}(n) \|_{\mathcal{X}} \leq 1$ in the definition of regularity, which excludes the trivial case where only the zero functional is regular. Fixed various typos and improved claritySubjects: Machine Learning (cs.LG); Optimization and Control (math.OC); Machine Learning (stat.ML)
We study the approximation properties and optimization dynamics of recurrent neural networks (RNNs) when applied to learn input-output relationships in temporal data. We consider the simple but representative setting of using continuous-time linear RNNs to learn from data generated by linear relationships. Mathematically, the latter can be understood as a sequence of linear functionals. We prove a universal approximation theorem of such linear functionals, and characterize the approximation rate and its relation with memory. Moreover, we perform a fine-grained dynamical analysis of training linear RNNs, which further reveal the intricate interactions between memory and learning. A unifying theme uncovered is the non-trivial effect of memory, a notion that can be made precise in our framework, on approximation and optimization: when there is long term memory in the target, it takes a large number of neurons to approximate it. Moreover, the training process will suffer from slow downs. In particular, both of these effects become exponentially more pronounced with memory - a phenomenon we call the "curse of memory". These analyses represent a basic step towards a concrete mathematical understanding of new phenomenon that may arise in learning temporal relationships using recurrent architectures.
- [35] arXiv:2302.10538 (replaced) [pdf, html, other]
-
Title: Lasserre Hierarchy for Graph Isomorphism and Homomorphism IndistinguishabilityComments: Full version. 36 pages, 6 figuresJournal-ref: TheoretiCS, Volume 3 (2024), Article 20, 1-42Subjects: Combinatorics (math.CO); Computational Complexity (cs.CC); Discrete Mathematics (cs.DM); Logic in Computer Science (cs.LO); Optimization and Control (math.OC)
We show that feasibility of the $t^\text{th}$ level of the Lasserre semidefinite programming hierarchy for graph isomorphism can be expressed as a homomorphism indistinguishability relation. In other words, we define a class $\mathcal{L}_t$ of graphs such that graphs $G$ and $H$ are not distinguished by the $t^\text{th}$ level of the Lasserre hierarchy if and only if they admit the same number of homomorphisms from any graph in $\mathcal{L}_t$. By analysing the treewidth of graphs in $\mathcal{L}_t$, we prove that the $3t^\text{th}$ level of Sherali--Adams linear programming hierarchy is as strong as the $t^\text{th}$ level of Lasserre. Moreover, we show that this is best possible in the sense that $3t$ cannot be lowered to $3t-1$ for any $t$. The same result holds for the Lasserre hierarchy with non-negativity constraints, which we similarly characterise in terms of homomorphism indistinguishability over a family $\mathcal{L}_t^+$ of graphs. Additionally, we give characterisations of level-$t$ Lasserre with non-negativity constraints in terms of logical equivalence and via a graph colouring algorithm akin to the Weisfeiler--Leman algorithm. This provides a polynomial time algorithm for determining if two given graphs are distinguished by the $t^\text{th}$ level of the Lasserre hierarchy with non-negativity constraints.
- [36] arXiv:2404.09215 (replaced) [pdf, html, other]
-
Title: Optimum Beamforming and Grating Lobe Mitigation for Intelligent Reflecting SurfacesComments: 12 pages, 16 figuresSubjects: Signal Processing (eess.SP); Optimization and Control (math.OC)
Ensuring adequate wireless coverage in upcoming communication technologies such as 6G is expected to be challenging. This is because user demands of higher datarate require an increase in carrier frequencies, which in turn reduce the diffraction effects (and hence coverage) in complex multipath environments. Intelligent reflecting surfaces have been proposed as a way of restoring coverage by adaptively reflecting incoming electromagnetic waves in desired directions. This is accomplished by judiciously adding extra phases at different points on the surface. In practice, these extra phases are only available in discrete quantities due to hardware constraints. Computing these extra phases is computationally challenging when they can only be picked from a discrete distribution, and existing approaches for solving this problem were either heuristic or based on evolutionary algorithms. We solve this problem by proposing fast algorithms with provably optimal solutions. Our algorithms have linear complexity, and are presented with rigorous proofs for their optimality. We show that the proposed algorithms exhibit better performance. We analyze situations when unwanted grating lobes arise in the radiation pattern, and discuss mitigation strategies, such as the use of triangular lattices and prephasing techniques, to eliminate them. We also demonstrate how our algorithms can leverage these techniques to deliver optimum beamforming solutions.
- [37] arXiv:2405.06190 (replaced) [pdf, other]
-
Title: Geometric Approaches to Matrix Normalization and Graph BalancingComments: V2: Updated Figure 1 and Remark 4.2, fixed typos, minor changes to expositionSubjects: Differential Geometry (math.DG); Optimization and Control (math.OC)
Normal matrices, or matrices which commute with their adjoints, are of fundamental importance in pure and applied mathematics. In this paper, we study a natural functional on the space of square complex matrices whose global minimizers are normal matrices. We show that this functional, which we refer to as the non-normal energy, has incredibly well-behaved gradient descent dynamics: despite it being non-convex, we show that the only critical points of the non-normal energy are the normal matrices, and that its gradient descent trajectories fix matrix spectra and preserve the subset of real matrices. We also show that, even when restricted to the subset of unit Frobenius norm matrices, the gradient flow of the non-normal energy retains many of these useful properties. This is applied to prove that low-dimensional homotopy groups of spaces of unit norm normal matrices vanish; for example, we show that the space of $d \times d$ complex unit norm normal matrices is simply connected for all $d \geq 2$. Finally, we consider the related problem of balancing a weighted directed graph -- that is, readjusting its edge weights so that the weighted in-degree and out-degree is the same at each node. We adapt the non-normal energy to define another natural functional whose global minima are balanced graphs and show that gradient descent of this functional always converges to a balanced graph, while preserving graph spectra and realness of the weights. Our results were inspired by concepts from symplectic geometry and Geometric Invariant Theory, but we mostly avoid invoking this machinery and our proofs are generally self-contained.
- [38] arXiv:2406.07224 (replaced) [pdf, html, other]
-
Title: Differentiability and Optimization of Multiparameter Persistent HomologyComments: 13 pages + 13 page appendix, 8 figures. v2: update references and fix typosJournal-ref: Proceedings of the 41st International Conference on Machine Learning, PMLR 235:43986-44011, 2024Subjects: Computational Geometry (cs.CG); Algebraic Topology (math.AT); Optimization and Control (math.OC)
Real-valued functions on geometric data -- such as node attributes on a graph -- can be optimized using descriptors from persistent homology, allowing the user to incorporate topological terms in the loss function. When optimizing a single real-valued function (the one-parameter setting), there is a canonical choice of descriptor for persistent homology: the barcode. The operation mapping a real-valued function to its barcode is differentiable almost everywhere, and the convergence of gradient descent for losses using barcodes is relatively well understood. When optimizing a vector-valued function (the multiparameter setting), there is no unique choice of descriptor for multiparameter persistent homology, and many distinct descriptors have been proposed. This calls for the development of a general framework for differentiability and optimization that applies to a wide range of multiparameter homological descriptors. In this article, we develop such a framework and show that it encompasses well-known descriptors of different flavors, such as signed barcodes and the multiparameter persistence landscape. We complement the theory with numerical experiments supporting the idea that optimizing multiparameter homological descriptors can lead to improved performances compared to optimizing one-parameter descriptors, even when using the simplest and most efficiently computable multiparameter descriptors.