# Backpropagation formula for simple ANN

I'm trying to solve an exercise. I'm asked to train a 2-layer neural network, given a small dataset. Below is a picture I drew to understand the architecture and derive the formula for backpropagation.

In this case I just have two input features and a bias unit. Moreover, $$h = w_1x_1 + w_2x_2 + w_3$$, $$\hat{y} = \sigma(h)$$, which is the output, and $$E = -y ln \hat{y} - (1-y) ln (1-\hat{y})$$, with $$y$$ being the real label.

If I want to compute the partial derivative of the error wrt $$w_1$$, I would do the following: $$\frac{\partial}{\partial w_1} E = \frac{\partial}{\partial \hat{y}} \frac{\partial}{\partial h} \hat{y} \frac{\partial}{\partial w_1} h.$$

This should simply be $$-(y - \hat{y}) x_1$$. Nonetheless, in the skeleton I was given, the update for the weights reads:

$$W = W + \frac{\alpha}{m} (y-\hat{y}) \sigma ' (x) x,$$

where $$m$$ is just the number of examples. I do not understand where that $$\sigma ' (x)$$ comes from.

Moreover, the update for the weights is done after each epoch, and not after each training example, as I read in some textbooks. Is this another way to perform learning?

• You update the weights after each iteration. Epochs are just a programming tool and don't really have any statistical meaning. – generic_user Nov 17 '18 at 19:51