Prediction Interval , Confidence Interval , Standard error

Question

What is the relation between Confidence Interval and prediction interval and, standard error for a point estimate? Please explain with an example.

As per my understanding, the standard error is a variation of point estimate (of a parameter) around the average estimation of the point estimate.

This standard error is accounted in confidence intervals.

And Prediction interval carries many such confidence intervals.

Answer 1

A prediction interval is an interval that covers the future (or otherwise unknown) value of a random variable with some prespecified probability.

Prediction intervals are conceptually related to confidence intervals, but they are not the same. A prediction interval pertains to a realization (which has not yet been observed, but will be observed in the future), whereas a confidence interval pertains to a parameter (which is in principle not observable, e.g., the population mean).

In the time series context, prediction intervals are known as forecast intervals.

(This is copied verbatim from the tag wiki for the prediction-interval tag.)

---

The standard error is the estimated standard deviation of an estimate of a parameter. It is typically used to construct a confidence-interval for the parameter, together with the point estimate and usually a normal distribution, which is motivated by the Central Limit Theorem or similar.

---

As an example, let us simulate some weakly related data and fit a (correctly specified) model. In R:

set.seed(1)
nn <- 20
predictor <- runif(nn)
response <- 3*predictor+rnorm(nn)

model <- lm(response~predictor)
xx <- seq(min(predictor),max(predictor),by=.01)

plot(predictor,response,pch=19)
lines(xx,predict(model,newdata=data.frame(predictor=xx)))
)

We can now extract the quantities we are interested in:

• The standard error of (say) the parameter estimate for the regression slope is 0.7394:

  > summary(model)
  
  ... truncated ...
  
  Coefficients:
              Estimate Std. Error t value Pr(>|t|)  
  (Intercept)   0.9412     0.4593   2.049   0.0553 .
  predictor     1.4083     0.7394   1.905   0.0729 .
  

• This allows us to obtain a symmetric 95% confidence interval for the regression slope parameter of $[-0.14,2.96]$:

  > confint(model)
                    2.5 %   97.5 %
  (Intercept) -0.02380633 1.906270
  predictor   -0.14499836 2.961647
  

• Finally, we can get a 95% prediction interval for a new observation. For that we need to specify at what value of the predictor we want to calculate this PI. For instance, we could use a predictor value of $0.5$, yielding a PI of $[-0.34,3.63]$:

  > predict(model,newdata=data.frame(predictor=0.5),interval="prediction")
         fit        lwr      upr
  1 1.645394 -0.3415354 3.632323
  

We see how the standard error and the confidence interval pertain to unobservable parameters, while the prediction interval pertains to (yet unseen) observations.