# Gradient on subset of training data is proportional to the true gradient?

I have been thinking of proving the following:

Prove that the gradient calculated on a random subset of a training set on average is proportional to the true gradient.

However, proving is not my strongest competence and I am trying to get better. Intuitively it makes sense. How would I go on proving this?

• What have you tried?
– Tim
Sep 28, 2019 at 15:47
• I was thinking of looking at the subset S of training set T, $S \in T$, then find the gradient of the subset by some loss function $\nabla L(w_S)$ where $L(w_S) = \sum_{i \in S}||y(x_i)-t_i||^2_2$ and then show $E(\nabla L(w_S)) \propto \nabla L(w)$ where $\nabla L(w)$ is the true gradient of the training set. Something like that. Sep 28, 2019 at 17:40