# When are genetic algorithms a good choice for optimization?

Genetic algorithms are one form of optimization method. Often stochastic gradient descent and its derivatives are the best choice for function optimization, but genetic algorithms are still sometimes used. For example, the antenna of NASA's ST5 spacecraft was created with a genetic algorithm:

When are genetic optimization methods a better choice than more common gradient descent methods?

Genetic algorithms (GA) are a family of heuristics which are empirically good at providing a decent answer in many cases, although they are rarely the best option for a given domain.

You mention derivative-based algorithms, but even in the absence of derivatives there are plenty of derivative-free optimization algorithms that perform way better than GAs. See this and this answer for some ideas.

What many standard optimization algorithms have in common (even derivative-free methods) is the assumption that the underlying space is a smooth manifold (perhaps with a few discrete dimensions), and the function to optimize is somewhat well-behaved.

However, not all functions are defined on a smooth manifold. Sometimes you want to optimize over a graph or other discrete structures (combinatorial optimization) -- here there are dedicated algorithms, but GAs would also work.

The more you go towards functions defined over complex, discrete structures, the more GAs can be useful, especially if you can find a representation in which the genetic operators work at their best (which requires a lot of hand-tuning and domain knowledge).

Of course, the future might lead to forget GAs altogether and develop methods to map discrete spaces to continuous space, and use the optimization machinery we have on the continuous representation.

• Anecdotally, I've found a simple, homemade genetic algorithm to outperform both Bayesian optimization and random search on media mix modeling task. May 3, 2021 at 20:36

Genetic methods are well suited for multicriteria optimization when gradient descent is dedicated to monocriteria optimization. Gradient descent allow to find minimum of functions when derivatives exists and there is only one optimum solution (if we except local minimas). A genetics algorithm can be used in multicriteria problems and lead to a continuum of solutions, each one beeing individuals of a population, having evolved from a initial population. The values to optimize are the phenotypes of the individuals and there can be several phenotypes. Generaly, none of the individual has simultaneously the better value of each phenotype, so there is not only one solution. The individuals in the final population, that are all solutions of the optimization, are part of the "Pareto front" and marked as being "Pareto rank one" individuals. This mean that compared to every others individuals having the same performance for each phenotype, they are at least better for one phenotype than the others.

• Ok for a downvote, but could you explain where I am wrong ? Dec 3, 2016 at 20:39
• This site values answers that provide context and background. See this help page for how to provide answers that will add to our repository of useful answers to interesting questions. Explaining your answer is also a good way to test your own understanding. For example, in this case you might want to expand on how genetic algorithms "are well suited for multicriteria optimization," as the Wikipedia page seems to imply single-valued fitness functions as objectives for genetic algorithms.
– EdM
Dec 3, 2016 at 21:16

Best in which sense ?

In my experience, GAs are one of the most pragmatic optimizers. While many more precise algorithms require time and effort to formalize real problems in the mathematical world, GAs can handle any cost function with complex rules and constraints (GAs are related by an execution approach afterall and not by specific calculation). This process is straightforward and you can try many approaches for exploratory work.

I appreciate also the possibility to reinject past solutions to the algorithm for future runs which is good for repeated task.

Conceptually, a genetic algorithm can be represented by a hashmap of functions and suits so functionnal languages well like Clojure which is also a language where you can achieve big results very quickly.

Genetic Algorithms can also be nested : the cost function of one GA can be a GA ! These algorithms take advantage of modern hardware and infrastructure which allow them to compute a very large population so that - even with simple mutation/selection operations - you can still achieve good results.

Even for simple problems like finding the minimum of a wave function, GAs are not that bad and can achieve a decent precision in an acceptable time.

So yeah, analytical solutions may have quicker execution time and precision, but the time required to produce them overweights often expected benefits ! So when ? Almost everytime to me, at least for meta-optimization.

• The main thrust of this argument seems to be that genetic algorithms are black-box optimizers. But there are plenty of black-box optimizers out there. Why would this be any better than other choices? In addition, I don't think it's really the case that GA's can easily handle constraints. For example, if the function is undefined except for a 3D subspace in a 4D world, certainly a vanilla GA would fail. Jun 10, 2017 at 0:50
• @CliffAB In fact I did not say anything about that and maybe more the contrary. In GA, you have a lot of control over core calculation, the GA in itself is a sequence of steps and light ordering. When you define cost functions, you can handle anything in the function, even external constraints you can query. My main arguments are : handle many problems, you have not to be concerned with the compatibility with the framework (you have just to return a cost), come up with a decent real-life solution in most business cases EVEN if it is not always the best Jun 15, 2017 at 11:47

Genetic algorithms are best when many processors can be used in parallel. and when the object function has a high modality (many local optima). Also, for multi-objective optimization, there are multi-objective genetic algorithms, MOGA.

However, I think Genetic algorithms are overrated. A lot of the popularity probably comes from the fact that they are inspired by biology. They do tend to find global optima more often than other direct search methods, but at a high cost, which means that another direct search method (e.g. Nelder-Mead, Complex or SQP) can be run several times to find a global optimum, and yield a better performance (i.e., in terms of convergence, and probability of finding global optimum) overall. This is true not only if a single processor is used, but also even when multi processors are available.

I think they stay popular because there are so many problems where very simple models are used, so performance is not an issue.