1
$\begingroup$

WITH an intercept, we have

$$ r = \hat{\beta} \frac{\sigma_x}{\sigma_y} \\ $$

Without, an intercept, is there a relationship between $r$ and $\hat{\beta}_{no intercept}$?

I know the folloowing

$$ \hat{\beta}_{no intercept} = \frac{\sum_i x_i y_i}{x_i^2} $$

It's not clear to me how this relates to $r$, where $$ r = \frac{\sum_i (x_i - \bar{x})(y_i - \bar{y})}{\sqrt{\sum_i (x_i - \bar{x})^2} \sqrt{\sum_i(y_i - \bar{y})^2}} $$

It's difficult to see how $\hat{\beta}_{nointercept}$ can be written as a function of $r$ in this case.

$\endgroup$
3
  • $\begingroup$ There really is no concept of $R^2$ in the no intercept case. web.ist.utl.pt/~ist11038/compute/errtheory/,regression/… $\endgroup$
    – mlofton
    Aug 19, 2020 at 2:23
  • $\begingroup$ I don't see why the correlation between observed response and that predicted by the line might not be useful. Under a different guise, it's called concordance correlation, measuring agreement, not linearity. It's just not "the" correlation. $\endgroup$
    – Nick Cox
    Aug 19, 2020 at 12:16
  • $\begingroup$ I'm sorry. I didn't mean that correlation was not useful. I meant that the actual calculation of $R^2$ that results from a regression ( which is 1 - sse/sstot) doesn't hold because the sum of squares decomposition ( sstot = ssr + ssse ) no longer holds. Apologies for confusion. I'm pretty sure you can calculate correlation between the two variables and it was still have meaning. It's the R^2 from the regression that has no meaning. $\endgroup$
    – mlofton
    Aug 19, 2020 at 13:13

1 Answer 1

1
$\begingroup$

In general, there does not appear to be any relationship. As the OP notes, the correlation coefficient has the same sign as the usual regression slope when the intercept is included, but the slope in the no intercept model can be opposite. Here is an example:

x = c(72, 72, 70, 69, 72, 64, 72, 69, 69, 67, 70, 76, 71, 72) 
y = c(6, 11, 7, 10, 12, 19, 10, 16, 9, 8, 5, 4, 7, 10)
cor(x,y)   # Negative 
fit = lm(y ~ x)  
summary(fit)     # Negative and significant slope
fit.noint = lm(y ~ x-1)
summary(fit.noint)   # Positive and significant slope

The correlation and the ordinary slope are negative (and "significant"), while the no-intercept slope is positive (and "significant").

The difference is easily explained by the fact that the no intercept model actually forces the intercept to be zero, which can be seen graphically as follows:

slope.noint = fit.noint$coefficients
plot(x,y, main="Regression Fits With and Without Intercept")
abline(lsfit(x, y), lwd=2)
abline(0, slope.noint, lwd=2, col="red")
legend("topright", c("Fit with Int", "Fit w/o Int"), lwd=c(2:2), 
    col = c("black", "red"))

With and without intercept 1

To see more clearly what is going on, it is helpful to expand the axes so that the origin (0,0) is included. The no intercept line must pass through the origin.

plot(x,y, xlim = c(0,100), ylim = c(0,20), 
    main="Regression Fits With and Without Intercept")
abline(lsfit(x, y), lwd=2)
abline(0, slope.noint, lwd=2, col="red")
legend("topleft", c("Fit with Int", "Fit w/o Int"), lwd=c(2:2), 
    col = c("black", "red"))

The resulting graph shows the difference more clearly:

With and without intercept 2

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.