As we know that we can perform a Bias Variance decomposition of an Estimator with MSE as loss function and it will look like below: $$MSE(\hat{\theta}) = tr(Var[\hat{\theta}]) + (||Bias[\hat{\theta}]||)^2$$ Similarly, if we want to perform a Bias Variance decomposition of an predictor with MSE as a loss function then we it will look like: $$MSE(\hat{y}|X) = Var[\hat{y}] + (||Bias[\hat{y}]||)^2 + Error $$ I am more curious to to know the philosophy to break down a estimator or a predictor into Variance and Bias term. Why not some other terms? It is more of a broad question of why we can think of breaking estimators and predictors into this form. Just thinking aloud we can break a predictor into known distribution plus and error term or an estimator into an know distribution of the sample and an error term. Please do correct me if I have some misunderstanding in terms of my thought process.