I am extremely new to neural networks.

I would just like to ask if there is a need to have hidden layers in a neural network.

I read off Quora that a neural network with 10 input nodes and 10 output nodes will have 100 parameters and 10 bias units.

If thats the case it means there are no hidden layers?


Yes that is correct, in that case the input is mapped through the output via a single weight matrix (10 x 10) and a bias of (10 x1).

If you choose your activation function as a sigmoid function then the Network that you are describing is equivalent to logistic regression.

  • $\begingroup$ Can you explain more why it becomes logistic regression? Maybe some equations or examples? $\endgroup$ – aceminer Nov 24 '15 at 16:04
  • $\begingroup$ Every output node $j$ in your network first computes $z_{j}= \sum_{i=1}^{10}W_{j, i}x_{i} + b_{j}$ and then returns $\frac{1}{1 + exp(-z_{j})}$, where $x_{i}$ is input $i$. Now if you remove the $j$ index from this equation then this literally becomes the input-output mapping defined by logistic regression. Therefore when using a sigmoid activation function every output node of the network you describe is in fact a logistic regression unit (each with possibly different weights). See en.wikipedia.org/wiki/Logistic_regression#Basics "Definition of the Logistic Function" for this map $\endgroup$ – Sjoerd Nov 24 '15 at 16:12
  • $\begingroup$ I ment en.wikipedia.org/wiki/… for the logistic mapping for a single variable $\beta$ $\endgroup$ – Sjoerd Nov 24 '15 at 16:18

Neural network without hidden layers is a mere logistic regression.

  • $\begingroup$ This is only true if the output is binary. $\endgroup$ – Vivek Subramanian Apr 15 '19 at 1:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.