Let's say that we have a three dimentional convex function with a minimum marked by a red dot. Can a full batch gradient in a blue point not directed to a red dot as drawn below?
Actually this question is about Andrew Ng's lection Gradient Descent With Momentum (C2W2L06). He said: "If you take one iteration of gradient descent either batch or mini-batch gradient descent maybe you end up heading [toward blue arrow]". Is it really possible for a full batch gradient?