I am working on a project where I need to implement the NEAT algorithm in python. after doing some research I came across an issue that I can't seem to find a solution for online, I hope this is the correct place to ask my question, if not then please refer me to where I can ask it.

From my understanding, the neural networks used in NEAT are "Partially Connected Neural Networks" meaning that there are no layers but rather only nodes, and each node can connect to every other node.

After trying to implement this process, I came across the issue of establishing by which rules should a new node be added and where, because there are no layers there is no definite solution I managed to come up with.

One solution I encountered is to add nodes on existing connections, something like the following image: enter image description here

But following this approach would only lead to a chain of single connected nodes, i.e. this would never lead to something like:

enter image description here

So to sum everything up, my question is, What are the rules for evolving a partially connected neural network using a genetic algorithm?


1 Answer 1


I believe that in the original NEAT implementation, new nodes were added on an existing connection, as you described in the first scenario. This is useful as it allows you to add a new node without modifying the output of the network, as long as your weights are chosen well.

You could also duplicate an existing connection, and add a node on that connection. This would be a more disruptive change to the network leading to a potentially greater change in the output. You could adjust the weights of both the original and the duplicated connections to keep the output the same, however.

In my experience with evolutionary algorithms, it is good to have a balance of minimally disruptive to maximally disruptive mutation operators in order to balance the degree of exploration of the overall space of solutions with the degree of exploitation of the local space of existing solutions. You could have both of the above options as well as various others - add node to existing connection, duplicate connection, change connection, add connection, delete connection, etc. Usually the more disruptive changes are more productive at early stages of evolution and minimally disruptive are more productive at later stages. Essentially exploring the structure early and then optimizing the weights later. Much will also depend upon how you craft your fitness function, so keep that in mind as you develop new mutation operators.

  • $\begingroup$ Thank you for your answer. Is there an official name for those operators? When implementing it I want to try and keep it as formal as possible $\endgroup$
    – Tomergt45
    Commented Sep 23, 2020 at 9:58
  • $\begingroup$ I would say they are "mutation operators". $\endgroup$
    – KirkD_CO
    Commented Sep 23, 2020 at 13:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.