0
$\begingroup$

I'm quite new to Statistics and I'm facing a problem.

Is there any relation between $R^2$ and the correlation matrix of the covariates?

A short example is (case with 2 covariates) :

A7 ~ A1 + A2

cor(A7, A1) = 0.447777
cor(A7, A2) = -0.2116495
cor(A1, A2) = -0.6117879
summary(lm(A7 ~ A1 + A2)$r.squared = 0.2067062

Can we predict the $R^2$ with those values? I know that it's easy with one single covariate (square of the correlation but I don't know in the general case). Maybe it's not possible...

$\endgroup$
2
$\begingroup$

In a linear regression with two predictiors,

$$y \sim 1 + x_1 + x_2 + \epsilon,$$

you can use this formula to calculate $R^2$:

$$R^2 = \frac{r_{yx_1}^2 + r_{yx_2}^2 - 2 r_{yx_1} r_{yx_2} r_{x_1x_2}}{1 - r_{x_1x_2}^2},$$

where $r_{yx_1}$ is the correlation coefficient between $y$ and $x_1$, $r_{yx_2}$ is the correlation coefficient between $y$ and $x_2$, and $r_{x_1x_2}$ is the correlation coefficient between $x_1$ and $x_2$.


An example in R:

# Create random numbers
set.seed(1)
y <- rnorm(100)
x1 <- y + rnorm(100, sd = 0.2)
x2 <- y - x1 + rnorm(100, sd = 0.3)

# Compute correlations
r_yx1 <- cor(y, x1)
r_yx1
# [1] 0.9779927
r_yx2 <- cor(y, x2)
r_yx2
# [1] 0.01581697    
r_x1x2 <- cor(x1, x2)
r_x1x2
# [1] -0.1003943

# Fit linear regression and extract R-squared
fit <- lm(y ~ x1 + x2)    
summary(fit)$r.squared
# [1] 0.9695985

# Calculate R-squared based on the correlation coefficients and the formula above    
(r_yx1 ^ 2 + r_yx2 ^ 2 - 2 * r_yx1 * r_yx2 * r_x1x2) / (1 - r_x1x2 ^ 2)
# [1] 0.9695985

As you can see, the analytical result matches the one of the model fit.

$\endgroup$
  • $\begingroup$ Thank you very much for this clear answer! Do you know if there exists a formula for the general case (for N covariates) ? $\endgroup$ – Philippe Remy Feb 26 '15 at 14:09

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.