I am assuming you are asking for some intuition behind stochastic gradient decent (SGD) and not for a rigorous proof.
First, recall that you are actually only moving into the direction of the batch gradient scaled by the rate parameter α. Thus, for every batch you only change the parameters slightly.
Let us consider a simple example. Suppose you have to find the coordinates (x, y) of a point r in the 2D plane that is close to two given points p and q. More precisely, you have to find (x, y) that minimizes the function
f(x, y) = (x - pₓ)² + (y - pᵧ)² + (x - qₓ)² + (y - qᵧ)² .
The solution of this problem, is just the midpoint between p and q. (which is 1/2 (p+q)). Let us assume, however, we would be solving this problem using SGD.
Let the first batch be just the point p and the second batch just the point q. Hence, the first batch reduces the term
(x - pₓ)² + (y - pᵧ)²
while the second one reduces
(x - qₓ)² + (y - qᵧ)² .
The minimum value for the first term is the point p while the minimum value for the second term is q. (The gradients the two problems are 2 (r-p) and 2 (r-q), respectively.) Hence, if we apply SDG, we start with a random value for r. Then we move r slightly into the direction of p by applying the first batch (assuming that α is small enough). Then we move r slightly into the direction of q. These two steps are repeated.
The point r will move along a zig-zag path. First, getting a bit closer to p then a bit closer to q then a bit closer to p and so on. If you choose the rate α appropriately then r will actually move towards the midpoint between p and q, for the following reason. If you draw a line from the starting point to the midpoint of p and q. On average you will be moving into the direction of this line, since whatever one step moves into the direction perpendicular to this line will be canceled by the next step. An example is shown below.
From this example, you can see, which assumptions are made when using SGD. The parameter α has to be chosen small enough such that you are not jumping around wildly, but just move slightly towards the minimum of each batch. If you were to compute the corrections of all batches at once with respect to the same value of r, instead of updating r between each batch, and then applying all corrections at once, you would actually be performing a step of (batch) gradient decent. If α is small, then r does not change significantly when the correction of a single batch is applied. Hence, the effect of computing the corrections of all batches at once is very similar to computing the corrections after each update, and thus (batch) gradient decent and SDG converge to the same local minimum.
