In his [machine learning lectures](https://class.coursera.org/ml-003/lecture/62) (1 min 30 sec), Andrew Ng seems to estimate the bias using the training set error. Why is it ok to do it?

The definition of "bias" in machine learning (see [wiki](https://en.wikipedia.org/wiki/Bias%E2%80%93variance_tradeoff) or any references from it), at least for the mean squared error cost function, is the expected error when using as a prediction the *expected prediction* made by various training sets. So why would the error (for a given point) for a single training set be anywhere close to the error (for that point) when using the average of all possible training sets?