Return to Question

As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below:

$$MSE(\hat{\theta}) = tr(Var[\hat{\theta}]) + (||Bias[\hat{\theta}]||)^2$$

Similarly, if we want to perform a Bias Variance decomposition of an predictor with MSE as a loss function then we it will look like:

$$MSE(\hat{y}|X) = Var[\hat{y}] + (||Bias[\hat{y}]||)^2 + \sigma_{\epsilon}^2 $$

I am more curious to to know the philosophy to break down a estimator or a predictor into Variance and Bias term. Why not some other terms? It is more of a broad question of why we can think of breaking estimators and predictors into this form.

Just thinking aloud we can break a predictor into known distribution plus and error term or an estimator into an know distribution of the sample and an error term.

Please do correct me if I have some misunderstanding in terms of my thought process.

The paper which triggered this question in my head (Bit unrelated): https://faculty.wharton.upenn.edu/wp-content/uploads/2012/04/Strong.pdf

Updated:

• Edit 1: Predictor Error with $\sigma_{\epsilon}^2$
• Edit 2: Updated reference paper

As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below:

$$\operatorname{MSE}(\hat{\theta}) = \operatorname{tr}(\operatorname{Var}[\hat{\theta}]) + (\|{\operatorname{Bias}[\hat{\theta}]}\|)^2$$

Similarly, if we want to perform a Bias Variance decomposition of an predictor with MSE as a loss function then we it will look like:

$$\operatorname{MSE}(\hat{y}\mid X) = \operatorname{Var}[\hat{y}] + (\|{\operatorname{Bias}[\hat{y}]}\|)^2 + \sigma_{\varepsilon}^2 $$

I am more curious to to know the philosophy to break down a estimator or a predictor into Variance and Bias term. Why not some other terms? It is more of a broad question of why we can think of breaking estimators and predictors into this form.

Just thinking aloud we can break a predictor into known distribution plus and error term or an estimator into an know distribution of the sample and an error term.

Please do correct me if I have some misunderstanding in terms of my thought process.

The paper which triggered this question in my head (Bit unrelated): https://faculty.wharton.upenn.edu/wp-content/uploads/2012/04/Strong.pdf

Updated:

• Edit 1: Predictor Error with $\sigma_\varepsilon^2$
• Edit 2: Updated reference paper

As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below:

$$MSE(\hat{\theta}) = tr(Var[\hat{\theta}]) + (||Bias[\hat{\theta}]||)^2$$

Similarly, if we want to perform a Bias Variance decomposition of an predictor with MSE as a loss function then we it will look like:

$$MSE(\hat{y}|X) = Var[\hat{y}] + (||Bias[\hat{y}]||)^2 + \sigma_{var}^2 $$

I am more curious to to know the philosophy to break down a estimator or a predictor into Variance and Bias term. Why not some other terms? It is more of a broad question of why we can think of breaking estimators and predictors into this form.

Just thinking aloud we can break a predictor into known distribution plus and error term or an estimator into an know distribution of the sample and an error term.

Please do correct me if I have some misunderstanding in terms of my thought process.

Updated:

• Predictor Error with $\sigma_{var}^2$

As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below:

$$MSE(\hat{\theta}) = tr(Var[\hat{\theta}]) + (||Bias[\hat{\theta}]||)^2$$

Similarly, if we want to perform a Bias Variance decomposition of an predictor with MSE as a loss function then we it will look like:

$$MSE(\hat{y}|X) = Var[\hat{y}] + (||Bias[\hat{y}]||)^2 + \sigma_{\epsilon}^2 $$

I am more curious to to know the philosophy to break down a estimator or a predictor into Variance and Bias term. Why not some other terms? It is more of a broad question of why we can think of breaking estimators and predictors into this form.

Just thinking aloud we can break a predictor into known distribution plus and error term or an estimator into an know distribution of the sample and an error term.

Please do correct me if I have some misunderstanding in terms of my thought process.

The paper which triggered this question in my head (Bit unrelated): https://faculty.wharton.upenn.edu/wp-content/uploads/2012/04/Strong.pdf

Updated:

• Edit 1: Predictor Error with $\sigma_{\epsilon}^2$
• Edit 2: Updated reference paper

As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below:

$$MSE(\hat{\theta}) = tr(Var[\hat{\theta}]) + (||Bias[\hat{\theta}]||)^2$$

Similarly, if we want to perform a Bias Variance decomposition of an predictor with MSE as a loss function then we it will look like:

$$MSE(\hat{y}|X) = Var[\hat{y}] + (||Bias[\hat{y}]||)^2 + Error $$

I am more curious to to know the philosophy to break down a estimator or a predictor into Variance and Bias term. Why not some other terms? It is more of a broad question of why we can think of breaking estimators and predictors into this form.

Just thinking aloud we can break a predictor into known distribution plus and error term or an estimator into an know distribution of the sample and an error term.

Please do correct me if I have some misunderstanding in terms of my thought process.

As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below:

$$MSE(\hat{\theta}) = tr(Var[\hat{\theta}]) + (||Bias[\hat{\theta}]||)^2$$

Similarly, if we want to perform a Bias Variance decomposition of an predictor with MSE as a loss function then we it will look like:

$$MSE(\hat{y}|X) = Var[\hat{y}] + (||Bias[\hat{y}]||)^2 + \sigma_{var}^2 $$

I am more curious to to know the philosophy to break down a estimator or a predictor into Variance and Bias term. Why not some other terms? It is more of a broad question of why we can think of breaking estimators and predictors into this form.

Just thinking aloud we can break a predictor into known distribution plus and error term or an estimator into an know distribution of the sample and an error term.

Please do correct me if I have some misunderstanding in terms of my thought process.

Updated:

• Predictor Error with $\sigma_{var}^2$