Hidden layers in neural network

Question

I am extremely new to neural networks.

I would just like to ask if there is a need to have hidden layers in a neural network.

I read off Quora that a neural network with 10 input nodes and 10 output nodes will have 100 parameters and 10 bias units.

If thats the case it means there are no hidden layers?

Sjoerd · Accepted Answer · 2015-11-24 16:00:15Z

4

Yes that is correct, in that case the input is mapped through the output via a single weight matrix (10 x 10) and a bias of (10 x1).

If you choose your activation function as a sigmoid function then the Network that you are describing is equivalent to logistic regression.

answered Nov 24, 2015 at 16:00

Sjoerd

1,23312 silver badges14 bronze badges

$\begingroup$ Can you explain more why it becomes logistic regression? Maybe some equations or examples? $\endgroup$
– aceminer
Nov 24, 2015 at 16:04
$\begingroup$ Every output node $j$ in your network first computes $z_{j}= \sum_{i=1}^{10}W_{j, i}x_{i} + b_{j}$ and then returns $\frac{1}{1 + exp(-z_{j})}$, where $x_{i}$ is input $i$. Now if you remove the $j$ index from this equation then this literally becomes the input-output mapping defined by logistic regression. Therefore when using a sigmoid activation function every output node of the network you describe is in fact a logistic regression unit (each with possibly different weights). See en.wikipedia.org/wiki/Logistic_regression#Basics "Definition of the Logistic Function" for this map $\endgroup$
– Sjoerd
Nov 24, 2015 at 16:12
$\begingroup$ I ment en.wikipedia.org/wiki/… for the logistic mapping for a single variable $\beta$ $\endgroup$
– Sjoerd
Nov 24, 2015 at 16:18

Add a comment |

Yurii · Accepted Answer · 2015-11-24 15:55:03Z

1

Neural network without hidden layers is a mere logistic regression.

answered Nov 24, 2015 at 15:55

Yurii

1,8841 gold badge16 silver badges26 bronze badges

1

$\begingroup$ This is only true if the output is binary. $\endgroup$
– Vivek Subramanian
Apr 15, 2019 at 1:51

Add a comment |

2 Answers 2