2
$\begingroup$

Imagine you have a Neural Network with Sigmoids. It has an input $x$ and so a node would output $\tanh x$ to a connection. The connection would then output $w*tanhx$ where $w$ is the weight of the connection.

The problem is, what if an input is 0 and the desire output should be something like 1? Well if the input is 0, the output of the connection would be $w*tanh0 = 0$. So if the input is 0, then then output will always be 0 no matter how many nodes or connections you add or how much the weights on the connections are.

How would you make a simple network where an input of 0 will give you something other than 0?

$\endgroup$

1 Answer 1

6
$\begingroup$

That's why you need the bias term $b$, then the output will be like $tanh(wx+b)$.
When the input is 0, the output will be $tanh(b)$ instead of 0.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.