Skip to main content
Cross Validated

Return to Question

Tweeted twitter.com/StackStats/status/1266610311366348806
added 13 characters in body
Source Link
Josh
Josh
  • 4.6k
  • 5
  • 34
  • 59

The Bias Variance decomposition is a decomposition of an expectation, but I fail to follow what's actually assumed random specifically in this decomposition.

Take the specific regression example below in Eq. 7.9 from ESL. Is this expectation:

  1. across multiple fits of the model? (i.e. across multiple trainings?)
  2. across different data points x?

I get the impression that it's case #1, but that would be odd. The data is the data that we have for fitting, so what type of fitfits would thisthey be? 

E.g. is this looking at the generalization error? (forthat is, for any dataset that we train with, not (not necessarily the one that we have,) what $\text{Err}(x_0)$ we would see for a given generic point $x_0$? Or something else?

The Bias Variance decomposition is a decomposition of an expectation, but I fail to follow what's actually assumed random specifically in this decomposition.

Take the specific regression example below in Eq. 7.9 from ESL. Is this expectation:

  1. across multiple fits of the model? (i.e. across multiple trainings?)
  2. across different data points x?

I get the impression that it's case #1, but that would be odd. The data is the data that we have for fitting, so what type of fit would this be? E.g. is this looking at the generalization error? (for any dataset that we train with, not necessarily the one that we have, what $\text{Err}(x_0)$ we would see for a given generic point $x_0$? Or something else?

The Bias Variance decomposition is a decomposition of an expectation, but I fail to follow what's actually assumed random specifically in this decomposition.

Take the specific regression example below in Eq. 7.9 from ESL. Is this expectation:

  1. across multiple fits of the model? (i.e. across multiple trainings?)
  2. across different data points x?

I get the impression that it's case #1, but that would be odd. The data is the data that we have for fitting, so what type of fits would they be? 

E.g. is this looking at the generalization error? that is, for any dataset that we train with (not necessarily the one that we have) what $\text{Err}(x_0)$ we would see for a given generic point $x_0$? Or something else?

Source Link
Josh
Josh
  • 4.6k
  • 5
  • 34
  • 59

Bias-Variance decomposition: Expectations over what?

The Bias Variance decomposition is a decomposition of an expectation, but I fail to follow what's actually assumed random specifically in this decomposition.

Take the specific regression example below in Eq. 7.9 from ESL. Is this expectation:

  1. across multiple fits of the model? (i.e. across multiple trainings?)
  2. across different data points x?

I get the impression that it's case #1, but that would be odd. The data is the data that we have for fitting, so what type of fit would this be? E.g. is this looking at the generalization error? (for any dataset that we train with, not necessarily the one that we have, what $\text{Err}(x_0)$ we would see for a given generic point $x_0$? Or something else?