Genetic algorithms are one form of optimization method. Often stochastic gradient descent and its derivatives are the best choice for function optimization, but genetic algorithms are still sometimes used. For example, the antenna of NASA's ST5 spacecraft was created with a genetic algorithm:

ST5 antenna

When are genetic optimization methods a better choice than more common gradient descent methods?

up vote 16 down vote accepted

Genetic algorithms (GA) are a family of heuristics which are empirically good at providing a decent answer in many cases, although they are rarely the best option for a given domain.

You mention derivative-based algorithms, but even in the absence of derivatives there are plenty of derivative-free optimization algorithms that perform way better than GAs. See this and this answer for some ideas.

What many standard optimization algorithms have in common (even derivative-free methods) is the assumption that the underlying space is a smooth manifold (perhaps with a few discrete dimensions), and the function to optimize is somewhat well-behaved.

However, not all functions are defined on a smooth manifold. Sometimes you want to optimize over a graph or other discrete structures (combinatorial optimization) -- here there are dedicated algorithms, but GAs would also work.

The more you go towards functions defined over complex, discrete structures, the more GAs can be useful, especially if you can find a representation in which the genetic operators work at their best (which requires a lot of hand-tuning and domain knowledge).

Of course, the future might lead to forget GAs altogether and develop methods to map discrete spaces to continuous space, and use the optimization machinery we have on the continuous representation.

Genetic methods are well suited for multicriteria optimization when gradient descent is dedicated to monocriteria optimization. Gradient descent allow to find minimum of functions when derivatives exists and there is only one optimum solution (if we except local minimas). A genetics algorithm can be used in multicriteria problems and lead to a continuum of solutions, each one beeing individuals of a population, having evolved from a initial population. The values to optimize are the phenotypes of the individuals and there can be several phenotypes. Generaly, none of the individual has simultaneously the better value of each phenotype, so there is not only one solution. The individuals in the final population, that are all solutions of the optimization, are part of the "Pareto front" and marked as being "Pareto rank one" individuals. This mean that compared to every others individuals having the same performance for each phenotype, they are at least better for one phenotype than the others.

  • Ok for a downvote, but could you explain where I am wrong ? – manu190466 Dec 3 '16 at 20:39
  • 5
    This site values answers that provide context and background. See this help page for how to provide answers that will add to our repository of useful answers to interesting questions. Explaining your answer is also a good way to test your own understanding. For example, in this case you might want to expand on how genetic algorithms "are well suited for multicriteria optimization," as the Wikipedia page seems to imply single-valued fitness functions as objectives for genetic algorithms. – EdM Dec 3 '16 at 21:16

Best in which sense ?

In my experience, GAs are one of the most pragmatic optimizers. While many more precise algorithms require time and effort to formalize real problems in the mathematical world, GAs can handle any cost function with complex rules and constraints (GAs are related by an execution approach afterall and not by specific calculation). This process is straightforward and you can try many approaches for exploratory work.

I appreciate also the possibility to reinject past solutions to the algorithm for future runs which is good for repeated task.

Conceptually, a genetic algorithm can be represented by a hashmap of functions and suits so functionnal languages well like Clojure which is also a language where you can achieve big results very quickly.

Genetic Algorithms can also be nested : the cost function of one GA can be a GA ! These algorithms take advantage of modern hardware and infrastructure which allow them to compute a very large population so that - even with simple mutation/selection operations - you can still achieve good results.

Even for simple problems like finding the minimum of a wave function, GAs are not that bad and can achieve a decent precision in an acceptable time.

So yeah, analytical solutions may have quicker execution time and precision, but the time required to produce them overweights often expected benefits ! So when ? Almost everytime to me, at least for meta-optimization.

  • The main thrust of this argument seems to be that genetic algorithms are black-box optimizers. But there are plenty of black-box optimizers out there. Why would this be any better than other choices? In addition, I don't think it's really the case that GA's can easily handle constraints. For example, if the function is undefined except for a 3D subspace in a 4D world, certainly a vanilla GA would fail. – Cliff AB Jun 10 '17 at 0:50
  • @CliffAB In fact I did not say anything about that and maybe more the contrary. In GA, you have a lot of control over core calculation, the GA in itself is a sequence of steps and light ordering. When you define cost functions, you can handle anything in the function, even external constraints you can query. My main arguments are : handle many problems, you have not to be concerned with the compatibility with the framework (you have just to return a cost), come up with a decent real-life solution in most business cases EVEN if it is not always the best – Joseph Yourine Jun 15 '17 at 11:47

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.