Skip to main content
deleted 73 characters in body
Source Link
Richard Hardy
  • 69.5k
  • 13
  • 126
  • 278

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$.

enter image description here

This looks like a very special case, as corresponding diagrams in machine learning textbooks are often asymmetric. See e.g. James et al. "Introduction to Statistical Learning" Figure 2.12:

enter image description here

I think the curious result of the initial picture rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

Question: What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ shouldcould be expected to hold (at least approximately) for the optimal model? Feel free to restrict your attention to a special case if a more general answer is too involved.

P.S. There is no need to address the question of whether the initial picture must hold in general, as that may lead this thread off track. The answer is clearly negative, as indicated above and in Stephen Kolassa's answer.

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$.

enter image description here

This looks like a very special case, as corresponding diagrams in machine learning textbooks are often asymmetric. See e.g. James et al. "Introduction to Statistical Learning" Figure 2.12:

enter image description here

I think the curious result of the initial picture rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

Question: What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ should be expected to hold for the optimal model? Feel free to restrict your attention to a special case if a more general answer is too involved.

P.S. There is no need to address the question of whether the initial picture must hold in general, as that may lead this thread off track. The answer is clearly negative, as indicated above and in Stephen Kolassa's answer.

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$.

enter image description here

This looks like a very special case, as corresponding diagrams in machine learning textbooks are often asymmetric. See e.g. James et al. "Introduction to Statistical Learning" Figure 2.12:

enter image description here

I think the curious result of the initial picture rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

Question: What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ could be expected to hold (at least approximately) for the optimal model?

P.S. There is no need to address the question of whether the initial picture must hold in general, as that may lead this thread off track. The answer is clearly negative, as indicated above and in Stephen Kolassa's answer.

Tweeted twitter.com/StackStats/status/1372518160465858566
added 536 characters in body
Source Link
Richard Hardy
  • 69.5k
  • 13
  • 126
  • 278

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$.

enter image description here

This looks like a very special case, as corresponding diagrams in machine learning textbooks are often asymmetric. See e.g. James et al. "Introduction to Statistical Learning" Figure 2.12:

enter image description here

I think the curious result of the initial picture rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

Question: What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ should be expected to hold for the optimal model? Feel free to restrict your attention to a special case if a more general answer is too involved.

enter image description hereP.S. There is no need to address the question of whether the initial picture must hold in general, as that may lead this thread off track. The answer is clearly negative, as indicated above and in Stephen Kolassa's answer.

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$. I think the result rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

Question: What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ should be expected to hold for the optimal model? Feel free to restrict your attention to a special case if a more general answer is too involved.

enter image description here

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$.

enter image description here

This looks like a very special case, as corresponding diagrams in machine learning textbooks are often asymmetric. See e.g. James et al. "Introduction to Statistical Learning" Figure 2.12:

enter image description here

I think the curious result of the initial picture rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

Question: What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ should be expected to hold for the optimal model? Feel free to restrict your attention to a special case if a more general answer is too involved.

P.S. There is no need to address the question of whether the initial picture must hold in general, as that may lead this thread off track. The answer is clearly negative, as indicated above and in Stephen Kolassa's answer.

added 14 characters in body; edited title
Source Link
Richard Hardy
  • 69.5k
  • 13
  • 126
  • 278

When will $\text{Variance}=\text{Bias}^2$ hold for anthe optimal model?

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$. I think the result rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

Question: What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ should be expected to hold for the optimal model? Feel free to restrict your attention to a special case if a more general answer is too involved.

enter image description here

When will $\text{Variance}=\text{Bias}^2$ hold for an optimal model?

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$. I think the result rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ should be expected to hold for the optimal model? Feel free to restrict your attention to a special case if a more general answer is too involved.

enter image description here

When will $\text{Variance}=\text{Bias}^2$ hold for the optimal model?

Let us consider the bias-variance decomposition in the context of model selection. The picture below suggests the optimal model (the one minimizing the expected squared prediction error) will have $\text{Variance}=\text{Bias}^2$. I think the result rests on the curves of variance and squared bias being convex and rather symmetric. The convexity is probably sensible, but I am not so sure about the approximate symmetry.

Question: What are some concrete settings (model classes and data generating processes) in which $\text{Variance}=\text{Bias}^2$ should be expected to hold for the optimal model? Feel free to restrict your attention to a special case if a more general answer is too involved.

enter image description here

edited tags
Link
gunes
  • 58.2k
  • 4
  • 50
  • 88
Loading
Source Link
Richard Hardy
  • 69.5k
  • 13
  • 126
  • 278
Loading