0
$\begingroup$

In bivariate linear regression is there a direct relationship between sample size $n$, coefficient of determination $r^2$ and $\sigma_\beta$ (the standard error of coefficient $\beta$)?

Assume data have been normalized so both target and predictor variable have $\sigma=1$.

Putting the question another way, does $\sigma_\beta$ tell me something different to $r^2$ or are they measures of the same thing? Or, is it possible to have a strong, certain but unreliable link between two variables (large $\beta$, small $\sigma_\beta$, but small $r^2$)?

(In multiple regression this doesn't apply as even with high $r^2$, $\sigma_\beta$ can indicate uncertainty as to which of the multiple predictors is causing the response).

EDIT

Just got this out of my software (without standardized data):

regression coeff 0.023
stderr of coeff 0.0046
p=0.000002

n=131
multiple r2=0.17
adjusted r2=0.16

predictor std=22.5
target std=2.24

Standardized coefficient is presumably

$0.023*22.5/2.24 = 0.23$.

If standardized coefficient is the same as correlation, then

$r^2 = 0.23^2 = 0.053$

...not the same as the software gave. What am I doing wrong?

$\endgroup$
  • $\begingroup$ Do you mean "coefficient of determination" for "correlation", & "multiple regression" for "multivariate regression"? $\endgroup$ – Scortchi Sep 6 '13 at 10:35
  • $\begingroup$ Yes. (My bad on $r^2$ but isn't multiple regression the same as multivariate regression?) $\endgroup$ – Sideshow Bob Sep 6 '13 at 10:45
  • 1
    $\begingroup$ No; multivariate regression means a multiple response (target or outcome or dependent variable). Having multiple predictors (independent variables) does not itself make a regression multivariate. $\endgroup$ – Nick Cox Sep 6 '13 at 10:47
  • $\begingroup$ Your question is puzzling. If the predictor in bivariate regression has been standardised, then its coefficient equals the correlation: this is in essence an inevitable consequence. If not, then not in general. The way to think of this is in terms of units of measurement or dimensional analysis. A correlation, and hence its square, has no units, but a regression coefficient has units (units of y)/(units of x). Standardising washes out both units and leaves you with dimensionless numbers. $\endgroup$ – Nick Cox Sep 6 '13 at 10:53
  • $\begingroup$ Ok so you are saying it's not possible, with standardized data, to have large $\beta$ and small $r^2$, because they are both the same thing - a measure of effect size. $\sigma_\beta$ meanwhile tells me the significance. Thanks btw, learning a few things here :) $\endgroup$ – Sideshow Bob Sep 6 '13 at 11:08
3
$\begingroup$

Here is a silly example from R (which I do not know well, but you can download it and it amounts to a lingua franca):

> y = c(23,32,45,54,67,75) 
> x = c(1,2,3,4,5,6)
> lm(y ~ x)

Call:
lm(formula = y ~ x)

Coefficients:
(Intercept)            x  
      11.93        10.69  

> cor(y, x)
[1] 0.998227
> sd(y)
[1] 20.02665
> sd(x)
[1] 1.870829
> 10.69 * sd(x) / sd(y)
[1] 0.9986273

There is some rounding error because I just took the printed result for the coefficient, but the principle is sound. Other software gives identical results.

$\endgroup$
  • $\begingroup$ That's great, and works for me too with my data. Must be a bug somewhere in my preprocessing script. $\endgroup$ – Sideshow Bob Sep 6 '13 at 12:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.