Examples in the Real World where Evolutionary Algorithms/Genetic Algorithms Outperform other Classes of Optimization Algorithms

Question

I have been trying to do some research to find out if there are certain industries/types of problems or even specific examples in applied research paper where Evolutionary Algorithms (e.g. Genetic Algorithm) have demonstrated to consistently outperform other classes of Optimization Algorithms (e.g. gradient based algorithms, stochastic gradient descent, etc.).

Doing some reading online, I have outlined the following advantages of Evolutionary Algorithms:

• Applications where the derivative of the loss function does not exist (e.g. contains "indicator functions", piecewise functions).

• Applications where derivative of the loss function exists, but is very costly to evaluate (e.g. I have heard talks in which gradient free optimization techniques were suggested for various problems involving reinforcement learning, e.g. https://www.youtube.com/watch?v=CZE86BPDqCI).

• Applications in Discrete Combinatorics/Optimization problems (this is kind of related to the first point, but imagine trying to optimize functions in which the inputs are a set of discrete objects and the output is a value associated with different inputs - for example: travelling salesman problem, knapsack optimization, scheduling, etc.)

• Evolutionary Algorithms are sometimes preferred for certain types problems such as "games", in which optimal strategies are developed by mutating and combining random strategies according to their performance with respect to some target (e.g. https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies , https://www.youtube.com/watch?v=OGHA-elMrxI)

In short, it seems like Evolutionary Algorithms are either preferred in situations where classical optimization techniques (e.g. stochastic gradient descent) fundamentally can not be used, or instances where a noisy and expensive-to-evaluate function needs to be optimized.

I have found some real world examples where Evolutionary Algorithms demonstrated success in discrete combinatorial optimization such as in the field of Operations Research - but I am still searching for examples of Evolutionary Algorithms being used in instances where the function required to be optimized is "noisy and expensive", i.e. instances where technically non-evolutionary algorithms could also have been used, but were not used because of these costs.

My Question: Can someone please recommend some references or discuss some examples where in general, the function needed to be optimized is usually "noisy" and "expensive to evaluate"?

Answer 1

It's still an open field of research, but there are many examples. Hyperparameter tuning is an example of a noisy and expensive function that can be handled by evolutionary algorithms. Neural architecture search is another example. You can also find a list of examples in this review paper, that mentions things like:

6.1.1 Variable and parameter optimization
6.1.2 Symbolic and polynomial regression
6.1.3 Automated code production
6.1.4 Regular expression
6.1.5 Circuit design
6.1.6 Code improvement and optimization
6.1.7 Simulator testing
6.1.8 Walking robot
6.1.9 Automated machine learning
6.2.1 Configuring neuromorphic computers
6.2.2 Forecasting financial markets
6.2.3 Predicting future city landscapes
6.2.4 Designing an optimized floor-plan
6.2.5 Antenna design
6.2.6 Defect identification of electron microscopy images

but again, this list is not exhaustive. The general answer is that genetic algorithms are a broad family of general-purpose optimization algorithms and we have many examples, from many domains, where they proved to work nicely.