I have been trying to do some research to find out if there are certain industries/types of problems or even specific examples in applied research paper where Evolutionary Algorithms (e.g. Genetic Algorithm) have demonstrated to consistently outperform other classes of Optimization Algorithms (e.g. gradient based algorithms, stochastic gradient descent, etc.).
Doing some reading online, I have outlined the following advantages of Evolutionary Algorithms:
Applications where the derivative of the loss function does not exist (e.g. contains "indicator functions", piecewise functions).
Applications where derivative of the loss function exists, but is very costly to evaluate (e.g. I have heard talks in which gradient free optimization techniques were suggested for various problems involving reinforcement learning, e.g. https://www.youtube.com/watch?v=CZE86BPDqCI).
Applications in Discrete Combinatorics/Optimization problems (this is kind of related to the first point, but imagine trying to optimize functions in which the inputs are a set of discrete objects and the output is a value associated with different inputs - for example: travelling salesman problem, knapsack optimization, scheduling, etc.)
Evolutionary Algorithms are sometimes preferred for certain types problems such as "games", in which optimal strategies are developed by mutating and combining random strategies according to their performance with respect to some target (e.g. https://en.wikipedia.org/wiki/Neuroevolution_of_augmenting_topologies , https://www.youtube.com/watch?v=OGHA-elMrxI)
In short, it seems like Evolutionary Algorithms are either preferred in situations where classical optimization techniques (e.g. stochastic gradient descent) fundamentally can not be used, or instances where a noisy and expensive-to-evaluate function needs to be optimized.
I have found some real world examples where Evolutionary Algorithms demonstrated success in discrete combinatorial optimization such as in the field of Operations Research - but I am still searching for examples of Evolutionary Algorithms being used in instances where the function required to be optimized is "noisy and expensive", i.e. instances where technically non-evolutionary algorithms could also have been used, but were not used because of these costs.
My Question: Can someone please recommend some references or discuss some examples where in general, the function needed to be optimized is usually "noisy" and "expensive to evaluate"?