Regression and Standard Deviation

Question

When I plot my Regression Line and the corresponding lines for 2 standard deviations, sometimes it happens, that no values lie outside the 2 standard deviations. Is that possible/correct or must there values outside 2 standard deviations per definition of the standard deviation? I am not sure about this.

Answer 1

OK, given the comment by @whuber, I've revised my answer to make it more comprehensive and hopefully informative.

By 95% confidence intervals (CI) I mean 95% point-wise CI-s. There are also simultaneous CI-s, which are in general wider than point-wise CI, but they will not be discussed here.

Consider a simple regression setting: $ E[ y| x, \beta] = x^T\hat{\beta}$.

Then for each value of $x$, the corresponding 95%CI can be read as a confidence that the interval will contain the population mean for the specific value of $x$.

In other words, a specific 95%CI corresponds to an imagined scenario of many times repeated data gathering/sampling and re-analysis, where in (about) 95% of cases the calculated mean will be contained within such CI.

We know that the conditional distribution, for each $x$, of the mean response is normal with 5% point mass outside the CI of approximately $\pm 2SD$-s (obviously, the CI is the 95%CI).

Imagine for a moment that at each of the $x$ values the response was measured a large number of times.

Then, it should be clear that in case where the repeated measurements are many, about 5% of the points will be outside the corresponding CI. Otherwise, that CI wouldn't be well calibrated.

Now, back to the original problem of (supposedly) not seeing points outside the CI often enough.

When there are a few response data points per $x$, the extreme ones (let's call them so, that is, outside 95% CI) will show rarely, but in a large number of repeated scenarios, this will still occur for about 5% of the points. This means that one should see about one extreme response point per about 20 response points (for a specific $x$).