I've recently seen a formula in a set of statistical learning videos that suppose we fit some model to some training data $f'(x)$ and let $(x_{0},y_{0})$ be a test observation drawn from the sample. If the true model is $Y=f(X)+\epsilon$ with $f(x)=E(Y|X=x)$ then,


What I don't understand is how the model $f'(x)$ can have a variance? Isn't this a model with no error term? I understand $Y$ can have a variance because there is an error term in involved but $f'(x)$ does not. So how are they defining this? Is it the fact that based on a set of $x$ points the $y$'s can change because of the term $\varepsilon$ therefore causing the parameters of our model $f'$ to change? So in my mind it's like saying right we are going to fit a quadratic model or say a linear one I know this a-priori. Now where $f(x_{0})$ actually lands on the $y$-axis is determined by the training data which itself is random due to the error term. Am I on the right track?



Browse other questions tagged or ask your own question.