I've read about NEAT/Evolutionary Artificial Neural Networks/Genetic Algorithms.

I understand the concept of choosing the fittest neural networks and breeding them to produce another one, but how exactly does this work?

Do you simply choose at random, weights from the parents, for the child network? Do you cross over the bias weights too?

How would this produce a network as fit as its parents, would it not just produce a new random network, because the weights have lost their correlation?

After trying this weight-swapping, I got very bad results.

What are the in-depth steps to evolving a feed-forward neural network?

Something like this

  • $\begingroup$ I think you'll find the answer to your questions if you do some reading on Genetic Algorithms. There are numerous mating operators to produce offsprings, it's not possible to know a priori what works best. Your mating operator is probably suboptimal and GA are generally hard to parametrise. $\endgroup$ – Digio Sep 13 '17 at 20:02

In my view, the question is not clear enough to be answered. "Evolving" can have different meanings in different context. Here are some examples.

  • One can say the model is "evolving" when the weights/parameters are changing. Think about the online learning algorithm (comparing to batch learning), the model and parameter will change with new streaming data. For example, suppose want want to build a classifier on cat vs. dogs. At beginning, we fit model with large number of cat, the model will more likely to predict on cat. But later, the data is dominated with dog. The model will "evolve" and more likely to produce dog.

  • Another possibility is the model has some memory and can model sequential data. Such as Recurrent neural network.

  • $\begingroup$ I'm asking how feed-forward networks are better using genetic algorithms. $\endgroup$ – Tobi Sep 13 '17 at 17:51
  • $\begingroup$ Something like this $\endgroup$ – Tobi Sep 13 '17 at 19:46

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.