# Gradient Descent Rule in feedforward ANN

I am having a hard time understanding the Gradient Descent Rule for learning in a feedforward ANN. In particular, how do we determine the initial weight vector, and how is this weight vector adjusted after each epoch?

From what I've read, I know that we first define some error function depending on the weights, and I think that we choose the initial weight to be the minimizer of this error function. Is this right?

Gradient descent applies updates of the form $$x^{(k+1)} = x^{(k)} - \eta \nabla f(x^{(k)})$$ where $${}^{(k)}$$ indicates that this is the $$k$$th iteration of the procedure and $$\eta$$ is the learning rate. Stochastic gradient descent only uses a fraction of the data to estimate $$\nabla f(x^{(k)})$$.
• The learning rate is $\eta$ (eta); $\nu$ (nu) doesn't appear in that equation. We hope that the update is closer to the minimum than when we started; however, there are lots of ways that this can go wrong. One example: stats.stackexchange.com/questions/367397/… Another example: stats.stackexchange.com/questions/364360/… – Sycorax May 24 at 16:37