Solving Mixed Integer Programs Using Neural Networks
Authors:
Vinod Nair,
Sergey Bartunov,
Felix Gimeno,
Ingrid von Glehn,
Pawel Lichocki,
Ivan Lobov,
Brendan O'Donoghue,
Nicolas Sonnerat,
Christian Tjandraatmadja,
Pengming Wang,
Ravichandra Addanki,
Tharindi Hapuarachchi,
Thomas Keck,
James Keeling,
Pushmeet Kohli,
Ira Ktena,
Yujia Li,
Oriol Vinyals,
Yori Zwols
Abstract:
Mixed Integer Programming (MIP) solvers rely on an array of sophisticated heuristics developed with decades of research to solve large-scale MIP instances encountered in practice. Machine learning offers to automatically construct better heuristics from data by exploiting shared structure among instances in the data. This paper applies learning to the two key sub-tasks of a MIP solver, generating…
▽ More
Mixed Integer Programming (MIP) solvers rely on an array of sophisticated heuristics developed with decades of research to solve large-scale MIP instances encountered in practice. Machine learning offers to automatically construct better heuristics from data by exploiting shared structure among instances in the data. This paper applies learning to the two key sub-tasks of a MIP solver, generating a high-quality joint variable assignment, and bounding the gap in objective value between that assignment and an optimal one. Our approach constructs two corresponding neural network-based components, Neural Diving and Neural Branching, to use in a base MIP solver such as SCIP. Neural Diving learns a deep neural network to generate multiple partial assignments for its integer variables, and the resulting smaller MIPs for un-assigned variables are solved with SCIP to construct high quality joint assignments. Neural Branching learns a deep neural network to make variable selection decisions in branch-and-bound to bound the objective value gap with a small tree. This is done by imitating a new variant of Full Strong Branching we propose that scales to large instances using GPUs. We evaluate our approach on six diverse real-world datasets, including two Google production datasets and MIPLIB, by training separate neural networks on each. Most instances in all the datasets combined have $10^3-10^6$ variables and constraints after presolve, which is significantly larger than previous learning approaches. Comparing solvers with respect to primal-dual gap averaged over a held-out set of instances, the learning-augmented SCIP is 2x to 10x better on all datasets except one on which it is $10^5$x better, at large time limits. To the best of our knowledge, ours is the first learning approach to demonstrate such large improvements over SCIP on both large-scale real-world application datasets and MIPLIB.
△ Less
Submitted 29 July, 2021; v1 submitted 23 December, 2020;
originally announced December 2020.
Exact and approximate moment closures for non-Markovian network epidemics
Authors:
Lorenzo Pellis,
Thomas House,
Matt J. Keeling
Abstract:
Moment-closure techniques are commonly used to generate low-dimensional deterministic models to approximate the average dynamics of stochastic systems on networks. The quality of such closures is usually difficult to asses and the relationship between model assumptions and closure accuracy are often difficult, if not impossible, to quantify. Here we carefully examine some commonly used moment clos…
▽ More
Moment-closure techniques are commonly used to generate low-dimensional deterministic models to approximate the average dynamics of stochastic systems on networks. The quality of such closures is usually difficult to asses and the relationship between model assumptions and closure accuracy are often difficult, if not impossible, to quantify. Here we carefully examine some commonly used moment closures, in particular a new one based on the concept of maximum entropy, for approximating the spread of epidemics on networks by reconstructing the probability distributions over triplets based on those over pairs. We consider various models (SI, SIR, SEIR and Reed-Frost-type) under Markovian and non-Markovian assumption characterising the latent and infectious periods. We initially study two special networks, namely the open triplet and closed triangle, for which we can obtain analytical results. We then explore numerically the exactness of moment closures for a wide range of larger motifs, thus gaining understanding of the factors that introduce errors in the approximations, in particular the presence of a random duration of the infectious period and the presence of overlapping triangles in a network. We also derive a simpler and more intuitive proof than previously available concerning the known result that pair-based moment closure is exact for the Markovian SIR model on tree-like networks under pure initial conditions. We also extend such a result to all infectious models, Markovian and non-Markovian, in which susceptibles escape infection independently from each infected neighbour and for which infectives cannot regain susceptible status, provided the network is tree-like and initial conditions are pure. This works represent a valuable step in deepening understanding of the assumptions behind moment closure approximations and for putting them on a more rigorous mathematical footing.
△ Less
Submitted 13 May, 2015;
originally announced May 2015.
