# Low Bias in an overfitted model

I have a question about the bias-variance tradeoff in machine learning concerning the implications of overfitting:

Assuming $$y = f(x)$$ + some noise, the error of our model for any input $$(x,y)$$ is given by: $$\begin{eqnarray} E [y-\hat{f}(x)] &=& (E[f(x)-\hat{f}(x)]^2 + E[(\hat{f}(x) - E[\hat{f}(x)])^2] \\ &=& \text{Bias}(\hat f(x),f(x))^2 + \text{Var}(\hat f(x)) \\ \end{eqnarray}$$

I understand that overfitting will lead to a high variance in our model, regarding random training input.

I also understand that we will have small bias terms for all $$(x_t, y_t)$$-pairs, that we used to train our model. But I don't see at all why an overfitted model will lead to a small bias in general, for example for some validation pair $$(x_v, y_v)$$. Actually the behaviour of an overfitted model, outside of the training data, seems completly unreliable to me.

Is the bias, that we are talking about in the "bias-variance tradeoff" only aiming at the bias for the given training data predictions?

I would understand that, but what I wonder about than is that regarding the variance only for a given training set doesn't seem to make any sense, since we are actually talking about the variance in our model for some random training input, that we used to generate our estimator, right?

2. $$\hat{f}$$, the estimated model, is a function of both y and x. Hence, it varies with different draws of the sample. $$E[\hat{f}]$$ is the average prediction of y using models estimated over various sample draws.