Skip to main content
3 of 5
added 14 characters in body; edited title
Richard Hardy
  • 69.5k
  • 13
  • 126
  • 278

When will $\text{Variance}=\text{Bias}^2$ hold for the optimal model?

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$. I think the result rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

Question: What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ should be expected to hold for the optimal model? Feel free to restrict your attention to a special case if a more general answer is too involved.

enter image description here

Richard Hardy
  • 69.5k
  • 13
  • 126
  • 278