Optimizing the ending rule of gradient method My intuition says that the third equation must be "the length of the gradient squared less than epsilon".
$x_{k+1} = x_k - f(x_k)$
$x_{k+1} = x_k + 1$
$|f(x_k)|^2 < \epsilon$  
However, I am not sure whether it is the standard form.
How would you write the standard form of the gradient method, and particularly its ending rule?
 A: Interpreting your function $f(x)$ as a (scaled) version of the gradient, your termination rule is equivalent to $|f(x)| < \sqrt{\epsilon}$, i.e. "terminate when you've taken a step that is too small." This seems perfectly reasonable. 
A: A simplified version of the gradient descent algorithm is as follows:
You begin with $k=0$, $x_k$, $\alpha_k$ and a backtracking parameter $c \in (0,1)$.
The descent direction is given by $p_k = - \nabla f(x_k)$.
Each iterate is computed as follows, $x_{k+1} = x_k + \alpha p_k$.
Remember that $\alpha$ is interpreted as step-length and must satisfy the sufficient descent condition, $f(x_k + p_k ) \leq f(x_k) + c \alpha_k \nabla f(x_k)^T p_k$. The value of $\alpha$ must be calculated using the backtracking procedure.
The algorithm is as follows:
data: k=0; x; alpha; c; tol
begin:
compute the gradient;
while( norm(grad) > tol ){
  compute alpha using backtracking;
  p = -grad;
  x = x + alpha*p;
}

The parameter tol is the tolerance for the stopping rule, it basically ensures that your gradient is close enough to zero. If the gradient is close to zero you can proof that your current iteration is close to a local minimum.
I hope this answers your question.
For further reference you may check Numerical Optimization by Nocedal & Wright.   
