Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It only takes a minute to sign up.
Sign up to join this community
I have a linear model $Y=\hat{\alpha}X+\hat{\beta}$ fit on a bunch of samples $(X_i,Y_i)$. How can I compute the prediction confidence interval for a given $X'$ ? Assuming than $X'$ does not belong to the sample set used to estimate the linear model.
Intuitively, the confidence interval will grow larger if $X'$ is "far away" from the training sample set. How does this intuition translate into equation ?
Many thanks !
The 95% prediction interval for a new value $Y^\prime = \hat\beta_0 + \hat\beta_1x^\prime$ (not used to find the regression line) corresponding to a new value $x^\prime$ is
$$Y^\prime \pm t_.975\,S_{Y|x}\sqrt{1 + \frac 1n + \frac{(x^\prime - \bar x)^2}{S_{xx}}},$$ where $\bar x$ is the mean of the $x_i,$ $S_{xx} = \sum(x_i - \bar x)^2$ (the numerator of the variance of the $x_i);$ $t_.975$ cuts probability 0.025 from the upper tail of $\mathsf{T}(\nu = n-2);$ and $S_{Y|x}^2 = \frac{1}{n-2}\sum_{i=1}^n (Y_i - \hat Y_i)^2$ (the residual variance). [The $x_i$ and $Y_i$ are data used to find the regression line.]
Formulas for this prediction interval (perhaps with slightly different notation) are shown in many elementary textbooks and online discussions about regression. [For example, Section 11.5 of Ott & Longnecker: Intro to statistical methods and data analysis, 7e.]
Now to discuss your second paragraph: Suppose that $x^\prime$ is within the span of the original data (roughly called 'interpolation'), so it is realistic to suppose that the regression model remains accurate. Then the increasing width of the prediction interval as $x^\prime$ becomes farther for $\bar x$ is governed by the third term inside the square-root sign. Notice that this distance is judged proportionately to the variability of the $x_i.$
Roughly speaking, the other two terms under the square root sign recognize the variability due to sampling a fresh observation and the variability due to error estimating the slope $\beta_0,$ respectively. [If you omit the $1$ inside the square root sign, then the interval becomes a 95% confidence interval for the height of the regression line at $x^\prime.]$
If the new value $x^\prime$ is beyond the span of the original $x_i,$ then this prediction may not be valid. (Roughly, called 'extrapolation'.) In practice, a frequent example is to have economic data up to last year $x_n$ and to try to use the corresponding regression model to predict the value of $Y^\prime$ corresponding to next year $x^\prime.$
