-
Bridging the Gap Between Theory and Practice: Benchmarking Transfer Evolutionary Optimization
Authors:
Yaqing Hou,
Wenqiang Ma,
Abhishek Gupta,
Kavitesh Kumar Bali,
Hongwei Ge,
Qiang Zhang,
Carlos A. Coello Coello,
Yew-Soon Ong
Abstract:
In recent years, the field of Transfer Evolutionary Optimization (TrEO) has witnessed substantial growth, fueled by the realization of its profound impact on solving complex problems. Numerous algorithms have emerged to address the challenges posed by transferring knowledge between tasks. However, the recently highlighted ``no free lunch theorem'' in transfer optimization clarifies that no single…
▽ More
In recent years, the field of Transfer Evolutionary Optimization (TrEO) has witnessed substantial growth, fueled by the realization of its profound impact on solving complex problems. Numerous algorithms have emerged to address the challenges posed by transferring knowledge between tasks. However, the recently highlighted ``no free lunch theorem'' in transfer optimization clarifies that no single algorithm reigns supreme across diverse problem types. This paper addresses this conundrum by adopting a benchmarking approach to evaluate the performance of various TrEO algorithms in realistic scenarios. Despite the growing methodological focus on transfer optimization, existing benchmark problems often fall short due to inadequate design, predominantly featuring synthetic problems that lack real-world relevance. This paper pioneers a practical TrEO benchmark suite, integrating problems from the literature categorized based on the three essential aspects of Big Source Task-Instances: volume, variety, and velocity. Our primary objective is to provide a comprehensive analysis of existing TrEO algorithms and pave the way for the development of new approaches to tackle practical challenges. By introducing realistic benchmarks that embody the three dimensions of volume, variety, and velocity, we aim to foster a deeper understanding of algorithmic performance in the face of diverse and complex transfer scenarios. This benchmark suite is poised to serve as a valuable resource for researchers, facilitating the refinement and advancement of TrEO algorithms in the pursuit of solving real-world problems.
△ Less
Submitted 20 April, 2024;
originally announced April 2024.
-
Preference Incorporation into Many-Objective Optimization: An Outranking-based Ant Colony Algorithm
Authors:
Gilberto Rivera,
Carlos A. Coello Coello,
Laura Cruz-Reyes,
Eduardo R. Fernandez,
Claudia Gomez-Santillan,
Nelson Rangel-Valdez
Abstract:
In this paper, we enriched Ant Colony Optimization (ACO) with interval outranking to develop a novel multiobjective ACO optimizer to approach problems with many objective functions. This proposal is suitable if the preferences of the Decision Maker (DM) can be modeled through outranking relations. The introduced algorithm (named Interval Outranking-based ACO, IO-ACO) is the first ant-colony optimi…
▽ More
In this paper, we enriched Ant Colony Optimization (ACO) with interval outranking to develop a novel multiobjective ACO optimizer to approach problems with many objective functions. This proposal is suitable if the preferences of the Decision Maker (DM) can be modeled through outranking relations. The introduced algorithm (named Interval Outranking-based ACO, IO-ACO) is the first ant-colony optimizer that embeds an outranking model to bear vagueness and ill-definition of DM preferences. This capacity is the most differentiating feature of IO-ACO because this issue is highly relevant in practice. IO-ACO biases the search towards the Region of Interest (RoI), the privileged zone of the Pareto frontier containing the solutions that better match the DM preferences. Two widely studied benchmarks were utilized to measure the efficiency of IO-ACO, i.e., the DTLZ and WFG test suites. Accordingly, IO-ACO was compared with two competitive many-objective optimizers: The Indicator-based Many-Objective ACO and the Multiobjective Evolutionary Algorithm Based on Decomposition. The numerical results show that IO-ACO approximates the Region of Interest (RoI) better than the leading metaheuristics based on approximating the Pareto frontier alone.
△ Less
Submitted 15 July, 2021;
originally announced July 2021.
-
AdaSwarm: Augmenting Gradient-Based optimizers in Deep Learning with Swarm Intelligence
Authors:
Rohan Mohapatra,
Snehanshu Saha,
Carlos A. Coello Coello,
Anwesh Bhattacharya,
Soma S. Dhavala,
Sriparna Saha
Abstract:
This paper introduces AdaSwarm, a novel gradient-free optimizer which has similar or even better performance than the Adam optimizer adopted in neural networks. In order to support our proposed AdaSwarm, a novel Exponentially weighted Momentum Particle Swarm Optimizer (EMPSO), is proposed. The ability of AdaSwarm to tackle optimization problems is attributed to its capability to perform good gradi…
▽ More
This paper introduces AdaSwarm, a novel gradient-free optimizer which has similar or even better performance than the Adam optimizer adopted in neural networks. In order to support our proposed AdaSwarm, a novel Exponentially weighted Momentum Particle Swarm Optimizer (EMPSO), is proposed. The ability of AdaSwarm to tackle optimization problems is attributed to its capability to perform good gradient approximations. We show that, the gradient of any function, differentiable or not, can be approximated by using the parameters of EMPSO. This is a novel technique to simulate GD which lies at the boundary between numerical methods and swarm intelligence. Mathematical proofs of the gradient approximation produced are also provided. AdaSwarm competes closely with several state-of-the-art (SOTA) optimizers. We also show that AdaSwarm is able to handle a variety of loss functions during backpropagation, including the maximum absolute error (MAE).
△ Less
Submitted 19 May, 2021; v1 submitted 19 May, 2020;
originally announced June 2020.
-
Enhancing Selection Hyper-heuristics via Feature Transformations
Authors:
I. Amaya,
J. C. Ortiz-Bayliss,
A. Rosales-Pérez,
A. E. Gutiérrez-Rodríguez,
S. E. Conant-Pablos,
H. Terashima-Marín,
C. A. Coello Coello
Abstract:
Hyper-heuristics are a novel tool. They deal with complex optimization problems where standalone solvers exhibit varied performance. Among such a tool reside selection hyper-heuristics. By combining the strengths of each solver, this kind of hyper-heuristic offers a more robust tool. However, their effectiveness is highly dependent on the 'features' used to link them with the problem that is being…
▽ More
Hyper-heuristics are a novel tool. They deal with complex optimization problems where standalone solvers exhibit varied performance. Among such a tool reside selection hyper-heuristics. By combining the strengths of each solver, this kind of hyper-heuristic offers a more robust tool. However, their effectiveness is highly dependent on the 'features' used to link them with the problem that is being solved. Aiming at enhancing selection hyper-heuristics, in this paper we propose two types of transformation: explicit and implicit. The first one directly changes the distribution of critical points within the feature domain while using a Euclidean distance to measure proximity. The second one operates indirectly by preserving the distribution of critical points but changing the distance metric through a kernel function. We focus on analyzing the effect of each kind of transformation, and of their combinations. We test our ideas in the domain of constraint satisfaction problems because of their popularity and many practical applications. In this work, we compare the performance of our proposals against those of previously published data. Furthermore, we expand on previous research by increasing the number of analyzed features. We found that, by incorporating transformations into the model of selection hyper-heuristics, overall performance can be improved, yielding more stable results. However, combining implicit and explicit transformations was not as fruitful. Additionally, we ran some confirmatory tests on the domain of knapsack problems. Again, we observed improved stability, leading to the generation of hyper-heuristics whose profit had a standard deviation between 20% and 30% smaller.
△ Less
Submitted 12 December, 2018;
originally announced December 2018.
