It should not.

[[1]] has shown that gradient descent with *random initialization* and *appropriate constant step size* does not converge to a saddle point. It is a long discussion but to give you an idea of why see the following example:

$$f(x,y)=\frac12 x^2+ \frac14y^4 - \frac12y^2$$

[![enter image description here][2]][2]

The critical points are
$$z_1=\begin{bmatrix}0\\0\end{bmatrix}, z_2=\begin{bmatrix}0\\1\end{bmatrix}, z_3=\begin{bmatrix}0\\-1\end{bmatrix}$$.

The points $z_2$ and $z_3$ are isolated local minima, and $z_1$ is a saddle point.

Gradient descent initialized from any point of the form $z_0=\begin{bmatrix}x\\0\end{bmatrix}$ converges to the saddle point $z_1$. Any other initial point either diverges or converges to a local minimum, so the stable set of $z_1$ is the $x$-axis, which is a zero measure set in $\mathbb{R}^2$. By computing the Hessian,
$$\nabla^2f(x,y)=\begin{bmatrix}1&&0\\0&&3y^2-1\end{bmatrix}$$

we find that $\nabla^2f(z_1)$ has one positive eigenvalue with eigenvector that spans the $x$-axis, thus agreeing with our above characterization of the stable set. **If the initial point is chosen randomly, there is zero probability
of initializing on the $x$-axis and thus zero probability of converging to the saddle point $z_1$**.



  [1]: https://arxiv.org/abs/1602.04915
  [2]: https://i.sstatic.net/Q6t1z.jpg