Showing 1–8 of 8 results for author: Sahinidis, N V

Transfer Learning in Information Criteria-based Feature Selection

Authors: Shaohan Chen, Nikolaos V. Sahinidis, Chuanhou Gao

Abstract: This paper investigates the effectiveness of transfer learning based on Mallows' Cp. We propose a procedure that combines transfer learning with Mallows' Cp (TLCp) and prove that it outperforms the conventional Mallows' Cp criterion in terms of accuracy and stability. Our theoretical results indicate that, for any sample size in the target domain, the proposed TLCp estimator performs better than t… ▽ More This paper investigates the effectiveness of transfer learning based on Mallows' Cp. We propose a procedure that combines transfer learning with Mallows' Cp (TLCp) and prove that it outperforms the conventional Mallows' Cp criterion in terms of accuracy and stability. Our theoretical results indicate that, for any sample size in the target domain, the proposed TLCp estimator performs better than the Cp estimator by the mean squared error (MSE) metric in the case of orthogonal predictors, provided that i) the dissimilarity between the tasks from source domain and target domain is small, and ii) the procedure parameters (complexity penalties) are tuned according to certain explicit rules. Moreover, we show that our transfer learning framework can be extended to other feature selection criteria, such as the Bayesian information criterion. By analyzing the solution of the orthogonalized Cp, we identify an estimator that asymptotically approximates the solution of the Cp criterion in the case of non-orthogonal predictors. Similar results are obtained for the non-orthogonal TLCp. Finally, simulation studies and applications with real data demonstrate the usefulness of the TLCp scheme. △ Less

Submitted 29 May, 2022; v1 submitted 6 July, 2021; originally announced July 2021.

Comments: Accepted to the Journal of Machine Learning Research

ACM Class: I.3; I.5

A Discussion on Practical Considerations with Sparse Regression Methodologies

Authors: Owais Sarwar, Benjamin Sauk, Nikolaos V. Sahinidis

Abstract: Sparse linear regression is a vast field and there are many different algorithms available to build models. Two new papers published in Statistical Science study the comparative performance of several sparse regression methodologies, including the lasso and subset selection. Comprehensive empirical analyses allow the researchers to demonstrate the relative merits of each estimator and provide guid… ▽ More Sparse linear regression is a vast field and there are many different algorithms available to build models. Two new papers published in Statistical Science study the comparative performance of several sparse regression methodologies, including the lasso and subset selection. Comprehensive empirical analyses allow the researchers to demonstrate the relative merits of each estimator and provide guidance to practitioners. In this discussion, we summarize and compare the two studies and we examine points of agreement and divergence, aiming to provide clarity and value to users. The authors have started a highly constructive dialogue, our goal is to continue it. △ Less

Submitted 8 February, 2021; v1 submitted 18 November, 2020; originally announced November 2020.

Comments: 12 pages

Journal ref: Statist. Sci. Volume 35, Number 4 (2020), 593-601

OR-Gym: A Reinforcement Learning Library for Operations Research Problems

Authors: Christian D. Hubbs, Hector D. Perez, Owais Sarwar, Nikolaos V. Sahinidis, Ignacio E. Grossmann, John M. Wassick

Abstract: Reinforcement learning (RL) has been widely applied to game-playing and surpassed the best human-level performance in many domains, yet there are few use-cases in industrial or commercial settings. We introduce OR-Gym, an open-source library for developing reinforcement learning algorithms to address operations research problems. In this paper, we apply reinforcement learning to the knapsack, mult… ▽ More Reinforcement learning (RL) has been widely applied to game-playing and surpassed the best human-level performance in many domains, yet there are few use-cases in industrial or commercial settings. We introduce OR-Gym, an open-source library for developing reinforcement learning algorithms to address operations research problems. In this paper, we apply reinforcement learning to the knapsack, multi-dimensional bin packing, multi-echelon supply chain, and multi-period asset allocation model problems, as well as benchmark the RL solutions against MILP and heuristic models. These problems are used in logistics, finance, engineering, and are common in many business operation settings. We develop environments based on prototypical models in the literature and implement various optimization and heuristic models in order to benchmark the RL results. By re-framing a series of classic optimization problems as RL tasks, we seek to provide a new tool for the operations research community, while also opening those in the RL community to many of the problems and challenges in the OR field. △ Less

Submitted 17 October, 2020; v1 submitted 14 August, 2020; originally announced August 2020.

Comments: 29 pages, 10 figures

MSC Class: 90-10 ACM Class: I.2.8

Domain reduction techniques for global NLP and MINLP optimization

Authors: Yash Puranik, Nikolaos V. Sahinidis

Abstract: Optimization solvers routinely utilize presolve techniques, including model simplification, reformulation and domain reduction techniques. Domain reduction techniques are especially important in speeding up convergence to the global optimum for challenging nonconvex nonlinear programming (NLP) and mixed-integer nonlinear programming (MINLP) optimization problems. In this work, we survey the variou… ▽ More Optimization solvers routinely utilize presolve techniques, including model simplification, reformulation and domain reduction techniques. Domain reduction techniques are especially important in speeding up convergence to the global optimum for challenging nonconvex nonlinear programming (NLP) and mixed-integer nonlinear programming (MINLP) optimization problems. In this work, we survey the various techniques used for domain reduction of NLP and MINLP optimization problems. We also present a computational analysis of the impact of these techniques on the performance of various widely available global solvers on a collection of 1740 test problems. △ Less

Submitted 26 June, 2017; originally announced June 2017.

Simulation optimization: A review of algorithms and applications

Authors: Satyajith Amaran, Nikolaos V. Sahinidis, Bikram Sharda, Scott J. Bury

Abstract: Simulation Optimization (SO) refers to the optimization of an objective function subject to constraints, both of which can be evaluated through a stochastic simulation. To address specific features of a particular simulation---discrete or continuous decisions, expensive or cheap simulations, single or multiple outputs, homogeneous or heterogeneous noise---various algorithms have been proposed in t… ▽ More Simulation Optimization (SO) refers to the optimization of an objective function subject to constraints, both of which can be evaluated through a stochastic simulation. To address specific features of a particular simulation---discrete or continuous decisions, expensive or cheap simulations, single or multiple outputs, homogeneous or heterogeneous noise---various algorithms have been proposed in the literature. As one can imagine, there exist several competing algorithms for each of these classes of problems. This document emphasizes the difficulties in simulation optimization as compared to mathematical programming, makes reference to state-of-the-art algorithms in the field, examines and contrasts the different approaches used, reviews some of the diverse applications that have been tackled by these methods, and speculates on future directions in the field. △ Less

Submitted 26 June, 2017; originally announced June 2017.

The ALAMO approach to machine learning

Authors: Zachary T. Wilson, Nikolaos V. Sahinidis

Abstract: ALAMO is a computational methodology for leaning algebraic functions from data. Given a data set, the approach begins by building a low-complexity, linear model composed of explicit non-linear transformations of the independent variables. Linear combinations of these non-linear transformations allow a linear model to better approximate complex behavior observed in real processes. The model is refi… ▽ More ALAMO is a computational methodology for leaning algebraic functions from data. Given a data set, the approach begins by building a low-complexity, linear model composed of explicit non-linear transformations of the independent variables. Linear combinations of these non-linear transformations allow a linear model to better approximate complex behavior observed in real processes. The model is refined, as additional data are obtained in an adaptive fashion through error maximization sampling using derivative-free optimization. Models built using ALAMO can enforce constraints on the response variables to incorporate first-principles knowledge. The ability of ALAMO to generate simple and accurate models for a number of reaction problems is demonstrated. The error maximization sampling is compared with Latin hypercube designs to demonstrate its sampling efficiency. ALAMO's constrained regression methodology is used to further refine concentration models, resulting in models that perform better on validation data and satisfy upper and lower bounds placed on model outputs. △ Less

Submitted 30 May, 2017; originally announced May 2017.

Optimization of circuitry arrangements for heat exchangers using derivative-free optimization

Authors: Nikolaos Ploskas, Christopher Laughman, Arvind U. Raghunathan, Nikolaos V. Sahinidis

Abstract: Optimization of the refrigerant circuitry can improve a heat exchanger's performance. Design engineers currently choose the refrigerant circuitry according to their experience and heat exchanger simulations. However, the design of an optimized refrigerant circuitry is difficult. The number of refrigerant circuitry candidates is enormous. Therefore, exhaustive search algorithms cannot be used and i… ▽ More Optimization of the refrigerant circuitry can improve a heat exchanger's performance. Design engineers currently choose the refrigerant circuitry according to their experience and heat exchanger simulations. However, the design of an optimized refrigerant circuitry is difficult. The number of refrigerant circuitry candidates is enormous. Therefore, exhaustive search algorithms cannot be used and intelligent techniques must be developed to explore the solution space efficiently. In this paper, we formulate refrigerant circuitry design as a binary constrained optimization problem. We use CoilDesigner, a simulation and design tool of air to refrigerant heat exchangers, in order to simulate the performance of different refrigerant circuitry designs. We treat CoilDesigner as a black-box system since the exact relationship of the objective function with the decision variables is not explicit. Derivative-free optimization (DFO) algorithms are suitable for solving this black-box model since they do not require explicit functional representations of the objective function and the constraints. The aim of this paper is twofold. First, we compare four mixed-integer constrained DFO solvers and one box-bounded DFO solver and evaluate their ability to solve a difficult industrially relevant problem. Second, we demonstrate that the proposed formulation is suitable for optimizing the circuitry configuration of heat exchangers. We apply the DFO solvers to 17 heat exchanger design problems. Results show that TOMLAB/glcDirect and TOMLAB/glcSolve can find optimal or near-optimal refrigerant circuitry designs after a relatively small number of circuit simulations. △ Less

Submitted 29 May, 2017; originally announced May 2017.

Computer-aided molecular design: An introduction and review of tools, applications, and solution techniques

Authors: Nick D. Austin, Nikolaos V. Sahinidis, Daniel W. Trahan

Abstract: This article provides an introduction to and review of the field of computer-aided molecular design (CAMD). It is intended to be approachable for the absolute beginner as well as useful to the seasoned CAMD practitioner. We begin by discussing various quantitative structure-property relationships (QSPRs) which have been demonstrated to work well with CAMD problems. The methods discussed in this ar… ▽ More This article provides an introduction to and review of the field of computer-aided molecular design (CAMD). It is intended to be approachable for the absolute beginner as well as useful to the seasoned CAMD practitioner. We begin by discussing various quantitative structure-property relationships (QSPRs) which have been demonstrated to work well with CAMD problems. The methods discussed in this article are (1) group contribution methods, (2) topological indices, and (3) signature descriptors. Next, we present general optimization formulations for various forms of the CAMD problem. Common design constraints are discussed and structural feasibility constraints are provided for the three types of QSPRs addressed. We then detail useful techniques for approaching CAMD optimization problems, including decomposition methods, heuristic approaches, and mathematical programming strategies. Finally, we discuss many applications that have been addressed using CAMD. △ Less

Submitted 14 January, 2017; originally announced January 2017.

Comments: 38 pages, 13 figures, 3 tables, 173 references

Journal ref: Chemical Engineering Research and Design, 116, 2-26, 2016