ISLR - Interpretation of Confidence Interval in Linear Regression

Question

I'm reading about Linear Regression in Introduction to Statistical Learning (Chapter 3) I see the confidence interval defined as

A 95% confidence interval is defined as range of values such that with 95% probability, the range will contain the true unknown value of the parameter

First off, this seems incorrect by various definitions I have seen, specifically wikipedia says

A 95% confidence level does not mean that for a given realized interval there is a 95% probability that the population parameter lies within the interval (i.e., a 95% probability that the interval covers the population parameter).

Unless authors of ISLR meant ... ranges of values ... instead of ... range of values ..., although that would be relying on reader noticing the plural word and being mindful of interpreting it that way.

It further goes on to interpret the computed confidence interval for the example advertising-sales data as (paraphrasing)

the 95% confidence interval for intercept ($\beta_0$) is [6.130, 7.935] . Therefore in the absence of any advertising, sales will on average fall somewhere between 6,130 and 7,940 units.

This again seems to be the interpretation that is incorrect.

More generally, I would also like to understand the utility of calculating a confidence interval (both for a general statistic and well as in the context of regression co-efficient)

The general interpretation seems to be that if we had 100 samples from population, and we calculated confidence intervals for the statistic using all 100 samples (independently), 95% of those intervals would have true the population statistic. i.e 95% denotes to the strength of the estimation method rather than that of the data.

Given that interpretation and the example interval for the sales-advertising example ([6.130, 7.935] ), what conclusions can I make about the interval?, I certainly couldn't say the true value of intercept is in that specific range 95% of the time (its either in that range or its not)? What insight does that specific range offer about the modeling procedure?

Answer 1

I would also like to understand the utility of calculating a confidence interval (both for a general statistic and well as in the context of regression co-efficient)

The utility in such estimators comes from giving us a range of possible values consistent with the data. Interpretation of the interval as such seems to be popular among a lot of practitioners.

The general interpretation seems to be that if we had 100 samples from population, and we calculated confidence intervals for the statistic using all 100 samples (independently), 95% of those intervals would have true the population statistic. i.e 95% denotes to the strength of the estimation method rather than that of the data.

You are right about the frequency properties of the interval. The "95%" in 95% confidence interval comes from the fact that the long term relative frequency of these estimators containing the true estimated is 95%.

Given that interpretation and the example interval for the sales-advertising example ([6.130, 7.935] ), what conclusions can I make about the interval?

Assuming the regression is in thousands of dollars, and that the interval provided is the for the additional revenue gained for thousand dollars spent on advertising, I would interpret it as follows:

"If we spent an additional thousand dollars on advertising, then from this data we should expect revenue of anywhere between \$6,130 to \$7, 935."

The key part of that is the "from this data" part. I think Frank Harrel and company talk a little about interpretation of confidence intervals here.

What insight does that specific range offer about the modeling procedure?

I don't think it offers insight about the modelling procedure per se, but rather about what is being modelled.

Answer 2

The interpretation of the confidence interval is slightly broader than described in the question.

Suppose a statistician carries out inference problems (eg. "impact of advertising on sales" described in the question) day in and day out on different populations (note that I'm not saying different samples of the same population) over many years, then, her confidence intervals will capture true population parameters 95% of the time.

Edited on @Sarvo 's comment:

How do we interpret the confidence interval? The confusion arises from the fact that the parameter being estimated is fixed. But the confidence interval is random (as it is a function of the sample ie. random variables. We can imagine a number line with parameter sitting at a fixed position, and, the confidence interval moving around it as we draw different samples from the population). So, we can make probability statements about the confidence interval. We say that the probability that the confidence interval traps the parameter value is 0.95.

A second edit:

Let the 95% confidence interval be $[X_L, X_H]$ for the parameter $\theta$. $X$ in $X_L$ and $X_H$ is to emphasize that they are random variables (as they are sample statistics ie. some function of the sample which is a group of random variables). Since it is the 95% confidence interval,

$P[X_L <\theta <X_H] = 0.95$

This can be also written as: $P[(X_L<\theta)\cap (X_H>\theta)] = 0.95$

The error is in interpreting this as a probability statement about $\theta$ which is not a random variable. We should interpret this as a probability statement about $X_L$ and $X_H$.