# Philosophical insight of Bias Variance Decomposition

As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below:

$$\operatorname{MSE}(\hat{\theta}) = \operatorname{tr}(\operatorname{Var}[\hat{\theta}]) + (\|{\operatorname{Bias}[\hat{\theta}]}\|)^2$$

Similarly, if we want to perform a Bias Variance decomposition of an predictor with MSE as a loss function then we it will look like:

$$\operatorname{MSE}(\hat{y}\mid X) = \operatorname{Var}[\hat{y}] + (\|{\operatorname{Bias}[\hat{y}]}\|)^2 + \sigma_{\varepsilon}^2$$

I am more curious to to know the philosophy to break down a estimator or a predictor into Variance and Bias term. Why not some other terms? It is more of a broad question of why we can think of breaking estimators and predictors into this form.

Just thinking aloud we can break a predictor into known distribution plus and error term or an estimator into an know distribution of the sample and an error term.

Please do correct me if I have some misunderstanding in terms of my thought process.

The paper which triggered this question in my head (Bit unrelated): https://faculty.wharton.upenn.edu/wp-content/uploads/2012/04/Strong.pdf

Updated:

• Edit 1: Predictor Error with $$\sigma_\varepsilon^2$$
• Edit 2: Updated reference paper
• Such a decomposition is a logical and nice consequence of the MSE. Indeed bias an variance of an estimator are atomic or elementary properties that we wish our estimator satisfies. Jan 24 at 20:13
• "I am more curious to to know the philosophy to break down a estimator or a predictor into Variance and Bias term." I've reread this several times, and I can't understand it. Jan 24 at 20:20
• As utobi said there are the elementary properties... @AdamO - If we want to make an estimator/predictor we need the atomic or elementary properties which will make the final estimaor... I am willing to know why we are sticking to bias and variance and what is the reasoning or thought process behind it and not something else. Jan 24 at 20:27
• $\hat{y}$ is a function of parameter estimates, in my experience it's kind of a trivial exercise to explore the properties of fitted values as a function of the properties of the regressors. But model specification, omitted variable bias, etc. are what you'd need to look into and it's a whole different topic/question. Jan 24 at 23:35
• Neither does the answer by @AdamO other than describing it as an elegant result. I think the question "why not do something another way" without proposing what shape this other way might take has no good answer. Jan 25 at 9:53