0
$\begingroup$

I have a linear model $Y=\hat{\alpha}X+\hat{\beta}$ fit on a bunch of samples $(X_i,Y_i)$. How can I compute the prediction confidence interval for a given $X'$ ? Assuming than $X'$ does not belong to the sample set used to estimate the linear model.

Intuitively, the confidence interval will grow larger if $X'$ is "far away" from the training sample set. How does this intuition translate into equation ?

Many thanks !

$\endgroup$
3
  • 1
    $\begingroup$ There are confidence intervals and prediction intervals. Which of the two are you interested in? $\endgroup$
    – Michael M
    Sep 23, 2019 at 18:04
  • $\begingroup$ I am quite sure he means prediction intervals actually prediction bands. Formulae for prediction bands & confidence bands in regression are well established under the standard assumption of independent normally distributed errors with 0 mean & constant variance. $\endgroup$ Sep 23, 2019 at 18:41
  • 1

1 Answer 1

2
$\begingroup$

The 95% prediction interval for a new value $Y^\prime = \hat\beta_0 + \hat\beta_1x^\prime$ (not used to find the regression line) corresponding to a new value $x^\prime$ is

$$Y^\prime \pm t_.975\,S_{Y|x}\sqrt{1 + \frac 1n + \frac{(x^\prime - \bar x)^2}{S_{xx}}},$$ where $\bar x$ is the mean of the $x_i,$ $S_{xx} = \sum(x_i - \bar x)^2$ (the numerator of the variance of the $x_i);$ $t_.975$ cuts probability 0.025 from the upper tail of $\mathsf{T}(\nu = n-2);$ and $S_{Y|x}^2 = \frac{1}{n-2}\sum_{i=1}^n (Y_i - \hat Y_i)^2$ (the residual variance). [The $x_i$ and $Y_i$ are data used to find the regression line.]

Formulas for this prediction interval (perhaps with slightly different notation) are shown in many elementary textbooks and online discussions about regression. [For example, Section 11.5 of Ott & Longnecker: Intro to statistical methods and data analysis, 7e.]

Now to discuss your second paragraph: Suppose that $x^\prime$ is within the span of the original data (roughly called 'interpolation'), so it is realistic to suppose that the regression model remains accurate. Then the increasing width of the prediction interval as $x^\prime$ becomes farther for $\bar x$ is governed by the third term inside the square-root sign. Notice that this distance is judged proportionately to the variability of the $x_i.$

Roughly speaking, the other two terms under the square root sign recognize the variability due to sampling a fresh observation and the variability due to error estimating the slope $\beta_0,$ respectively. [If you omit the $1$ inside the square root sign, then the interval becomes a 95% confidence interval for the height of the regression line at $x^\prime.]$

If the new value $x^\prime$ is beyond the span of the original $x_i,$ then this prediction may not be valid. (Roughly, called 'extrapolation'.) In practice, a frequent example is to have economic data up to last year $x_n$ and to try to use the corresponding regression model to predict the value of $Y^\prime$ corresponding to next year $x^\prime.$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.