0
$\begingroup$

I was just going through Neural-Network tutorials and I have some question regarding Bias neuron :-

1.] Is the act of introducing bias neuron same as introducing the X(index 0)=1 in the logistic regression i.e. [1, X1, X2]

2.] Is bias neuron an additional degree of freedom ?

3.] Can bias neuron be analogous to the constant term in the linear equation :-

Y= mX+C

because without C i can only rotate the line x around the origin, but if I take into consideration C also then I can translate this line up or down also.

For example consider the figure below :-

enter image description here

Then the blue line (hypothesis) is simply Y=m*X (blue line passes through origin), so even if you keep fiddling with the m paramater but you wont be able to seperate the green and blue dots ever with just this hypothesis. So with just m to fiddle with, my hypothesis will always have to pass through the origin, i.e. I can choose from the infinite numbers of lines through the origin by tweaking the parameter 'm'.

Whereas if I include C/constant term (which Im calling the bias) in my hypothesis then my equation becomes Y=m*X+C then a line like Y=.4*X+3 (magenta line ) easily classifies the data. So now that I have an additional parameter "C" , I can translate(shift up/shift down) apart from the rotation.

Is my understanding correct ?

Improve this question
$\endgroup$
2
  • $\begingroup$ Can you explain how will y always output .5 as per your answer in the link. $\endgroup$
    – shalini
    Jun 3, 2015 at 10:35
  • 1
    $\begingroup$ In case of zero input a sigmoid would always output .5 while you might want it not to fire or fire $1$. 1) yes it is the same as adding an extra dimension of ones to your data, 2) yes it is an extra degree of freedom as it is an extra parameter to be trained, 3) your intuition is correct... $\endgroup$
    – Yannis Assael
    Jun 3, 2015 at 16:13

0

Reset to default