After fitting a linear regression model on a sequence of observations we obtain the joint distribution of all our regression coefficients, given the training data.

Suppose now that we have an extr observation of the predictor variables; $x$, and want to determine a 95% CI for the corresponding $y$, given $x$ and all training data.

My idea is that $y$ is distributed as the inner product of the random vector of regression coefficients with $x$ plus a Gaussian noise which also has a random variance (according to the distribution of the estimated std error in the linear regression).

Alternatively, one can simply take the expected value of the regression coefficients, to obtain what one usually considers the coefficients, and compute the CI purely based on the Gaussian noise, this time with known variance (the estimated one).

Do these produce the same outcome? If not, which one is the more conceptually sound approach?

  • 1
    $\begingroup$ not identical: the second approach of course ignores uncertainty associated with the regression coefficients. If you would like your confidence intervals to cover the true value 95% of the time (which they indeed should by definition), then you will only find this to be the case under the first of your two proposed approaches. By the way, a confidence interval on a new data point is often given a special name, a prediction interval, which may help your googling. $\endgroup$ Commented Jul 2, 2023 at 20:47
  • 3
    $\begingroup$ The term confidence interval is usually reserved for unobservable parameters, whereas prediction interval pertains to future observables. Confusing the two is a common problem. See stats.stackexchange.com/tags/prediction-interval/info $\endgroup$ Commented Jul 2, 2023 at 23:15


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Browse other questions tagged or ask your own question.