Skip to main content
Cross Validated

Return to Answer

added 71 characters in body
Source Link
StatsML
StatsML
  • 786
  • 4
  • 10
  1. The expectation is with respect to draws of (y, x). So, when we are inside the expectation brackets, we are dealing with a given draw of (y, x).
  2. $\hat{f}$, the estimated model, is a function of both y and x. Hence, it varies with different draws of the sample. $E[\hat{f}]$ is the average prediction of y using models estimated over various sample draws.
  3. Thus, the bias term is a measure of "average" (over all the sample draws) of in-sample squared error (as we find the fitting error within the expectation ie. for a given sample).
  4. The variance on the other hand is the average of the difference between (a) model estimate from this sample and (b) the average of estimates of models fit in all other sample draws. This is the only place where we have a component of out-of-sample testing (through (b)). Thus, this is a measure of how much our estimates vary across samples.
  1. The expectation is with respect to draws of (y, x). So, when we are inside the expectation brackets, we are dealing with a given draw of (y, x).
  2. $\hat{f}$, the estimated model, is a function of both y and x. Hence, it varies with different draws of the sample. $E[\hat{f}]$ is the average prediction of y using models estimated over various sample draws.
  3. Thus, the bias term is a measure of "average" (over all the sample draws) of in-sample squared error.
  4. The variance on the other hand is the average of the difference between (a) model estimate from this sample and (b) the average of estimates of models fit in all other sample draws. This is the only place where we have a component of out-of-sample testing (through (b)). Thus, this is a measure of how much our estimates vary across samples.
  1. The expectation is with respect to draws of (y, x). So, when we are inside the expectation brackets, we are dealing with a given draw of (y, x).
  2. $\hat{f}$, the estimated model, is a function of both y and x. Hence, it varies with different draws of the sample. $E[\hat{f}]$ is the average prediction of y using models estimated over various sample draws.
  3. Thus, the bias term is a measure of "average" (over all the sample draws) of in-sample squared error (as we find the fitting error within the expectation ie. for a given sample).
  4. The variance on the other hand is the average of the difference between (a) model estimate from this sample and (b) the average of estimates of models fit in all sample draws. This is the only place where we have a component of out-of-sample testing (through (b)). Thus, this is a measure of how much our estimates vary across samples.
Source Link
StatsML
StatsML
  • 786
  • 4
  • 10

  1. The expectation is with respect to draws of (y, x). So, when we are inside the expectation brackets, we are dealing with a given draw of (y, x).
  2. $\hat{f}$, the estimated model, is a function of both y and x. Hence, it varies with different draws of the sample. $E[\hat{f}]$ is the average prediction of y using models estimated over various sample draws.
  3. Thus, the bias term is a measure of "average" (over all the sample draws) of in-sample squared error.
  4. The variance on the other hand is the average of the difference between (a) model estimate from this sample and (b) the average of estimates of models fit in all other sample draws. This is the only place where we have a component of out-of-sample testing (through (b)). Thus, this is a measure of how much our estimates vary across samples.