How does SGD training error decrease in subsequent epochs with non-iid samples when it is recommended that samples in subsequent epochs be iid? I have been reading the Deep Learning book by Ian Goodfellow and on pg. 277, they mention:

It is also crucial that the minibatches be selected randomly.
Computing an unbiased estimate of the expected gradient from a set of
samples requires that those samples be independent. We also wish for
two subsequent gradient estimates to be independent from each other,
so two subsequent minibatches of examples should also be independent
from each other

I understand that for any unbiased estimation we require the samples to be i.i.d. so that we ideally end up with a true representation of the underlying data and so the above statement makes sense. However in practice, the samples that SGD sees for the subsequent gradient update (the next epoch) are the same, and it still performs well in the sense that the training error decreases. The authors later mention :

...but of course, the additional epochs usually provide enough beneﬁt
due to decreased training error to oﬀset the harm they cause by
increasing the gap between training error and test error

I know this happens but if someone could explain to me why and how it happens perhaps from a statistical perspective, it would be great! Another way to put it: why does training error in SGD decrease in subsequent epochs even though the samples are not i.i.d anymore?
 A: The question in the title is not consistent with what Goodfellow writes. This passage

It is also crucial that the minibatches be selected randomly. Computing an unbiased estimate of the expected gradient from a set of samples requires that those samples be independent. We also wish for two subsequent gradient estimates to be independent from each other, so two subsequent minibatches of examples should also be independent from each other

says that the unbiased estimates are required to be independent. On the other hand, your title asks "How does SGD training error decrease in subsequent epochs when statistically, it requires that samples in subsequent epochs be i.i.d and they are not?"
Goodfellow is saying you can compute biased estimates, and sometimes those biased estimates are good enough to improve the model. There is no requirement to compute an unbiased update.
If it were possible, we'd like to have all of our updates to the model be unbiased, but we would need vastly more data (a new mini-batch at each training step). This isn't possible because data is scarce. So the question then becomes "How far can we get using the biased updates from re-using training data?"
Goodfellow is saying that biased updates arising from re-using training data improves the model on the training set. Additionally, if the model is not overfitting, then these updates also improve the model in general. In other words, bias exists when we re-use the training data in subsequent epochs, but the amount of bias is not so severe that it prevents the model from improving.
This is a common theme across the field of statistics -- we might prefer a less-than-ideal estimator simply because it is "good enough" to solve the problem. Ideal estimators, if/when they exist, may have infeasibly expensive costs associated with them, or otherwise be impractical to use. At the same time, if a less-than-ideal estimator is not "good enough" to solve a problem, then we'll either need to find an alternative that does, or accept worse results, or else abandon the effort.
This interpretation is consistent with what Goodfellow writes. The quotation is simply an empirical observation:

"...the additional epochs usually provide enough beneﬁt..."

In other words, there are cases when re-using the training data does not provide enough benefit. The obvious case is when the model is overfitting. On the other hand, you could decide to stop training the model after one epoch, but that model may exhibit under-fitting.
