0
$\begingroup$

I have been thinking of proving the following:

Prove that the gradient calculated on a random subset of a training set on average is proportional to the true gradient.

However, proving is not my strongest competence and I am trying to get better. Intuitively it makes sense. How would I go on proving this?

Improve this question
$\endgroup$
2
  • $\begingroup$ What have you tried? $\endgroup$
    – Tim ♦
    Sep 28, 2019 at 15:47
  • $\begingroup$ I was thinking of looking at the subset S of training set T, $S \in T$, then find the gradient of the subset by some loss function $\nabla L(w_S)$ where $L(w_S) = \sum_{i \in S}||y(x_i)-t_i||^2_2$ and then show $E(\nabla L(w_S)) \propto \nabla L(w)$ where $\nabla L(w)$ is the true gradient of the training set. Something like that. $\endgroup$
    – Chris
    Sep 28, 2019 at 17:40

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.