8
$\begingroup$

When doing a hypothesis test for whether some value $a$ is significantly different from some other value $b$, if the confidence interval for $a$ contains $b$, then we cannot reject the null hypothesis and thus $a$ is not significantly different from $b$.

Take the following graph for example (plotted with seaborn.regplot in python): Scatter plot with no significant trend

When I do a proper significance test on the regression I've fitted I get the following:

coef std err. t P> abs(t) [0.025 0.975]
const 0.2529 0.031 8.111 0.000 0.186 0.320
(1-log(momentum)) 0.0192 0.017 1.131 0.279 -0.017 0.056

The intercept is clearly significant but the slope is not significantly non-zero because its confidence interval contains 0.

When I was looking at the graph, I noticed that I would be able to draw a horizontal line that fits completely inside the confidence interval like this: Scatter plot with regression line and horizontal line through confidence interval

It seems like I might be able to tell whether the slope is significantly different from 0 just by looking at whether I can fit a horizontal line inside the confidence interval or not.

My question is this: Is this a coincidence or does this trick work in general? Does it also work for the intercept? (e.g if the x axis is inside the interval then it is not significantly different from 0)

$\endgroup$
7
  • 1
    $\begingroup$ I am curious to the answer to your question (I suspect something but I don’t KNOW it) but your example is not quite right. I would be very surprised if your residuals are normal distributed. Also there’s heteroscedasticity. Both violate assumptions in regression, so both are red flags in the analysis. $\endgroup$
    – W_vH
    Dec 18, 2023 at 22:03
  • 4
    $\begingroup$ I see little evidence of either non-Normality or heteroscedasticity in the residuals. // This is not a trick or a coincidence, but it needs a slightly different interpretation because the confidence band really depicts a confidence region for the pair of parameter estimates, which is not the same as a confidence interval for its slope alone. $\endgroup$
    – whuber
    Dec 18, 2023 at 22:10
  • 1
    $\begingroup$ @whuber So does this mean that if I can fit a horizontal line through the confidence region then there is at least one slope-intercept pair with slope=0 which are both inside the respective confidence intervals and thus that 0 is also inside the interval for the slope? Or is there a flaw in my reasoning? $\endgroup$ Dec 18, 2023 at 23:01
  • 1
    $\begingroup$ That sounds like a correct interpretation of a confidence region. $\endgroup$
    – whuber
    Dec 19, 2023 at 15:04
  • 3
    $\begingroup$ @ChristophHanck If these are prediction bands, they must be for something like a 50% level (at most) rather than a standard 95% level: look at how many of the data points lie beyond them. That is why I have interpreted them as confidence bands. $\endgroup$
    – whuber
    Dec 20, 2023 at 14:31

1 Answer 1

3
$\begingroup$

The estimate for a regression line is

$$\hat{y}|x = \hat{\alpha} + \hat{\beta}x$$

The variance of this estimate is

$$\text{Var}(\hat{y}|x) = \text{Var}(\hat{\alpha} ) + x^2 \text{Var}(\hat{\beta} ) + 2x Cov(\alpha,\beta)$$

And the standard deviation

$$SD(\hat{y}|x) = \sqrt{ \text{Var}(\hat{\alpha} ) + x^2 \text{Var}(\hat{\beta} ) + 2x Cov(\alpha,\beta)}$$

For $x=0$ the error of the regression line coincides with the error of the intercept coefficient $\alpha$

$$SD(\hat{y}|x=0) = SD(\hat{\alpha})$$

For large values of $x$ the error of the regression line coincides with the error of the slope coefficient $\beta$

$$SD(\hat{y}|x) \to x \text{SD}(\hat{\beta} ) $$

So the shape of the confidence interval will approach the error of the intercept when $x=0$ and the error of the slope for large magnitude of $x$. So if the confidence interval of the slope coefficient contains zero, then the confidence interval of the regression line will contain the horizontal line through the mean.

An edge case exception is when the covariance term $\text{cov}(\alpha,\beta)$ is negative and the bound of the confidence interval is exactly zero, then it will be possible that the horizontal line is outside the plotted confidence interval for the regression line.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.