According to this answer, a general rule of thumb is that your hidden layer size should be between your input and output sizes.

In developing my JavaScript neural network, this has proven to be about right. However, for boolean logic it seems to be wrong. For example, an xor task seems to need more than 2 hidden neurons in order to complete (I have tested this on my own net and it seems about right, as well as this other JS net here needing 3 neurons). Additionally, a subtraction task seems to need a great many as well, at least on my network.

Is there a separate rule of thumb for these types of logic tasks?

up vote 0 down vote accepted

The universal approximation theorem of neural networks states that a neural network with a bounded, continuous activation functions and given enough hidden units in only one hidden layer can approximate any function. So, in theory you should be able to get the correct output, but you do not know in advance what should be the right number of hidden units for your problem. So, you may not necessarily need to choose a number of hidden units less than the number of your input units (however that's usually the case in practice)

Also, what I'm seeing is your network is a Perceptron network. If that's the case, then your network is only able to learn linearly-separable data points and you cannot learn complex functions.

Sources

  • Thanks a lot! My neurons use a sigmoid activation function, which I was led to believe somewhere else on the net would allow the perceptron to learn non linearly separable data. – user24984 Jan 7 '16 at 13:25

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.