0
$\begingroup$

According to this answer, a general rule of thumb is that your hidden layer size should be between your input and output sizes.

In developing my JavaScript neural network, this has proven to be about right. However, for boolean logic it seems to be wrong. For example, an xor task seems to need more than 2 hidden neurons in order to complete (I have tested this on my own net and it seems about right, as well as this other JS net here needing 3 neurons). Additionally, a subtraction task seems to need a great many as well, at least on my network.

Is there a separate rule of thumb for these types of logic tasks?

Improve this question
$\endgroup$
0

1 Answer 1

Reset to default
0
$\begingroup$

The universal approximation theorem of neural networks states that a neural network with a bounded, continuous activation functions and given enough hidden units in only one hidden layer can approximate any function. So, in theory you should be able to get the correct output, but you do not know in advance what should be the right number of hidden units for your problem. So, you may not necessarily need to choose a number of hidden units less than the number of your input units (however that's usually the case in practice)

Also, what I'm seeing is your network is a Perceptron network. If that's the case, then your network is only able to learn linearly-separable data points and you cannot learn complex functions.

Sources

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thanks a lot! My neurons use a sigmoid activation function, which I was led to believe somewhere else on the net would allow the perceptron to learn non linearly separable data. $\endgroup$
    – user24984
    Jan 7, 2016 at 13:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.