In tutorials, gradient descent is often shown as a point descending down a bowl shaped error surface. As it learns from examples it descends down the surface towards the minima.
It struck me today though that the error surface is defined in a nueral network per training example. So the surface should appear different for each example. Why then do I never see the error surface morphing as the point descends downwards?
Has my intuition lost it's way somewhere?