2
$\begingroup$

I will do research using NN with 1 hidden layer. To calculate loss using binary cross entropy and for the activation function using sigmoid. I found the derivative formula from Sadowski, 2016 (link: https://www.ics.uci.edu/~pjsadows/notes.pdf) as follows:

enter image description here

A weight Wji connecting hidden layer unit j to output unit i has gradient:

enter image description here

enter image description here

The paper does not explain how the derivative of error with respect to bias. So, how is the formula for the derivative of error with respect to bias? Is there any other source that can explain it because I have to include the source. Thank you so much.

$\endgroup$
1
  • $\begingroup$ Does the answer by gunes answer your question? If so, consider accepting it to mark this question as solved. $\endgroup$
    – kmkurn
    Dec 31, 2022 at 5:19

1 Answer 1

3
$\begingroup$

For bias, you can take the input as $1$ in these equations, i.e. $x_k=1$. So, that will be

$$\frac{\partial E}{\partial b^1_j}=\sum_i (y_i-t_i)w_{ji}(h_j(1-h_j))$$

where $b^1_j$ is the bias term of neuron $j$ in the hidden layer.

$\endgroup$
2
  • $\begingroup$ Oh, I see. Thank you so much. Is that mean if there is bias bi, I can take hj =1? So derivative E/ derivative bi = (yi-ti)1 ? is that correct? $\endgroup$
    – Andryan
    Dec 26, 2022 at 23:16
  • $\begingroup$ Yes, that's correct $\endgroup$
    – gunes
    Dec 26, 2022 at 23:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.