3
$\begingroup$

We have a simple regression equation $y=a+bx$, where $a,b$ were estimated via OLS -- we know these values. Suppose the number of observations $N=25$ is given. Is it true, that we cannot calculate $R^2$ (or $F$-statistic, equivalently)? Equivalently, is it really impossible to calculate the standard error $\sigma_b$?

$\endgroup$
1
  • 3
    $\begingroup$ It is true that this information does not determine $R^2$ or $F:$ you need information about the correlation or the residuals or about $\sigma_b$ itself. Please search our site for regression formulas related to standard errors. $\endgroup$
    – whuber
    Commented Apr 3 at 15:28

3 Answers 3

5
$\begingroup$

I'll give a simpler answer.

𝑅2 is calculated from the distance of the line from the points, the variation among the values, and sample size. If all you know is the slope, intercept and sample size, you know nothing about how far the points are from the line or how much the values vary.

$\endgroup$
4
  • $\begingroup$ If all you know is the slope, intercept and sample size, you know nothing about how far the points are from the line.---Hmm that may not be necessarily true. If you at least have the descriptive data in a summary table, and see that they are both strongly correlated with very little variance, then one could guess to a degree what it could be, even if you don't have the physical data on hand. However this would still be an approximation rather than an exact estimation. $\endgroup$ Commented Apr 4 at 1:30
  • 1
    $\begingroup$ @ShawnHemelstrand. You are postulating that you know more than the slope, intercept and sample size! $\endgroup$ Commented Apr 4 at 3:50
  • $\begingroup$ Yes I suppose that means you would know more than is led on in this question. My comment was just a slight caveat. I anyway think your answer is still correct. $\endgroup$ Commented Apr 4 at 3:52
  • 1
    $\begingroup$ Although this is a good heuristic, the problem is that the meaning of "closeness" of a line to the points is vague and easily misinterpreted, because $R^2$ doesn't actually measure closeness in any standard sense. For instance, if you were to double all coordinates, $R^2$ would be unchanged but in any reasonable metric all distances would double, too. $\endgroup$
    – whuber
    Commented Apr 4 at 16:02
3
$\begingroup$

My answer is similar to the first, only worded differently. As a thought experiment, say you would also know the residuals and the values of X. Then you can calculate $R^2$ obviously. Then multiply the residuals by 10. This increases the unexplained variance and thus decreases the explained variance or $R^2$ value. However, the intercept and slope do not change. This can be seen as follows.

Based on the model, the observed Y values are equal to

$Y = a + b X + resid$

Becasuse $a$ and $b$ are the least-squares estimates, the residuals $resid$ sum to zero and the correlation of $X$ and $resid$ is also zero.

If the residuals would be multiplied by 10 we get new Y values equal to:

$Y_{new} = a + b X + 10*resid$

For this equation it also holds that the residuals $10*resid$ sum to zero (if $\sum{resid}=0$ then $\sum{(10*resid)}=10*\sum{resid}=0$) and that the correlation of the residuals $10*resid$ and $X$ is zero (multiplying a variable with a constant does not change the correlation of that variable with some other variable). Because both these conditions hold, $a$ and $b$ must also be the least-squares estimates of the regression of $Y_{new}$ on $X$.

$\endgroup$
2
  • $\begingroup$ +1 It's a great explanation -- but it would be nice to demonstrate the assertion at the end. (Hint: the normal equations still hold after the multiplication.) $\endgroup$
    – whuber
    Commented Apr 4 at 16:00
  • $\begingroup$ Yes @whuber, that is a good idea, I will do that, thanks. $\endgroup$
    – BenP
    Commented Apr 4 at 16:48
3
$\begingroup$

Let me give the following concrete (toy) example to illustrate @Harvey Motulsky's answer: Consider the following two datasets:

  • dat1: $N = 4$, $x_1 = -1, x_2 = -1, x_3 = 1, x_4 = 1$, $y_1 = -1, y_2 = 1, y_3 = 1, y_4 = 3$.
  • dat2: $N = 4$, $x_1 = -1, x_2 = -1, x_3 = 1, x_4 = 1$, $y_1 = -2, y_2 = 2, y_3 = 0, y_4 = 4$.

The plot below shows scatter plots of these two datasets, together with their common fitted regression line $y = 1 + x$.

enter image description here

It can be seen that dat1 and dat2 give the same regression coefficient estimates ($a = b= 1$), yet different $R^2$ ($R_1^2 = 0.5, R_2^2 = 0.2$, see R code below for details). This example shows that $R^2$ cannot be determined by just $N, a$ and $b$.

dat1 <- data.frame(x = c(-1, -1, 1, 1), y = c(-1, 1, 1, 3))
dat2 <- data.frame(x = c(-1, -1, 1, 1), y = c(-2, 2, 0, 4))
plot(dat1$x, dat1$y, xlim = c(-2, 2), ylim = c(-3, 5), xlab = 'x', ylab = 'y')
points(dat2$x, dat2$y, pch = 19)
abline(a = 1, b = 1, lty = "dashed")

m1 <- lm(y ~ x, data = dat1) 
coef(m1) # returns c(1, 1)
summary(m1)$r.squared # returns 0.5
m2 <- lm(y ~ x, data = dat2)
coef(m2) # returns c(1, 1)
summary(m2)$r.squared # returns 0.2
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.