3
$\begingroup$

I have a R code that runs lm function and get the summary. Summary

What is the meaning of multiple R squared? And is there any relationship between multiple R-squared and correlation coefficient? Then how can we explain between two variables using multiple R squared?

$\endgroup$
  • $\begingroup$ If you would have something more specific to your problem, you can also edit your question and elaborate on that $\endgroup$ – StupidWolf Jan 25 at 11:05
4
$\begingroup$

In the case of a simple linear regression model (ie one with a single predictor, say $X$, and a single outcome, say $Y$) such as you have here, multiple R squared can be interpreted as the percentage of variance in $Y$ that can be explained by $X$.

Correlation is a measure of the linear association between two variables. There are several ways in which it can be calculated, but in the case of Pearson correlation, multiple R squared will be equal to the square of the correlation between $Y$ and $X$. And, the estimated regression coefficient for $X$, when scaled by the ratio of the standard deviations of $X$ and $Y$ and then squared, will also be equal to the square of the correlation between $X$ and $Y$

> set.seed(15)
> N <- 100

> Y <- rnorm(N,10,2)
> X <- runif(N,5,10)

m0 <- lm(Y ~ X)

> summary(m0)$r.squared
[1] 0.007241825

> cor(X, Y)^2
[1] 0.007241825

> all.equal(summary(m0)$r.squared, cor(X, Y)^2)
[1] TRUE

> (coef(m0)[[2]] * sd(X) / sd(Y))^2
[1] 0.007241825
$\endgroup$
4
$\begingroup$

In general, these four quantities are the same giving three different ways to think about R squared.

  • multiple R squared
  • the variance of the fitted values divided by the variance of the dependent variable (i.e. the regression "explains" that proportion of the variance of the dependent variable)
  • the square of the correlation between the fitted values and the dependent variable
  • the proportional improvement in residual sum of squares from the null model to the fitted model. (The null model has the same dependent variable but only an intercept with no predictors.)

We show an R example where X1 and X2 are the predictors, Y is the dependent variable, fm is the regression object and fitted(fm) are the fitted values. (The fitted values are the values of Y predicted by the right hand side of the regression equation.) fm_null is the regression object of the null regression model (as described above) and deviance(fm) refers to the residual sum of squares of the indicated model.

X1 <- c(1, 1, 1, 1, 2, 2)
X2 <- c(1, 2, 3, 1, 2, 3)
Y <- c(1, 2, 3, 4, 5, 6)
fm <- lm(Y ~ X1 + X2) # regression of Y on X1 and X2
fm_null <- lm(Y ~ 1)  # null model

# these are the same

var(fitted(fm)) / var(Y) # ratio of variances
## [1] 0.7032967

cor(fitted(fm), Y)^2 # squared correlation
## [1] 0.7032967

summary(fm)$r.squared # multiple R squared
## [1] 0.7032967

1 - deviance(fm) / deviance(fm_null)  # improvement in residual sum of squares
## [1] 0.7032967

Single Predictor

We can repeat this with the data used in the question which has the special feature that there is only a single predictor:

library(MASS)

fm <- lm(bwt ~ lwt, birthwt)
fm_null <- lm(bwt ~ 1, birthwt)

var(fitted(fm)) / var(birthwt$bwt) # ratio of variances
## [1] 0.03449685

cor(fitted(fm), birthwt$bwt)^2 # squared correlation
## [1] 0.03449685

summary(fm)$r.squared # multiple R squared
## [1] 0.03449685

1 - deviance(fm) / deviance(fm_null)  # improvement in residual sum of squares
## [1] 0.03449685

In the special case of a single predictor we have additional equalities which we can add to the above list:

  • the square of the correlation between the dependent variable and that predictor equals each of the above.

  • the coefficient of the predictor times the ratio of the standard deviations of the dependent variable to the predictor equals the correlation so the square of all that equals the multiple R squared.

  • if we standardize the dependent variable and predictor to each have mean 0 and standard deviation 1 then the coefficient of the predictor equals the correlation between the dependent variable and the predictor so its square equals multiple R squared.

Thus we have these three additional ways to view R squared (6 in total) in the case of a single predictor.

cor(birthwt$lwt, birthwt$bwt)^2 # squared correlation
## [1] 0.03449685

(coef(fm)[[2]] * sd(birthwt$lwt) / sd(birthwt$bwt))^2
## [1] 0.03449685

fm0 <- lm(bwt ~ lwt, as.data.frame(scale(birthwt)))
coef(fm0)[[2]]^2
## [1] 0.03449685 
$\endgroup$
0
$\begingroup$

Multiple R-Squared explains the percentage variation in Y (dependent variable) that can be explained by the Xs (independent variables). If you add more X(s) to the model, the multiple R-squared will increase but that doesn't necessarily mean that the newly added variable(s) is/are contributing to the explanation of Y. The adjusted R-Squared, adjust for the amount of variable(s) you've added to the model. This tells you if the newly added variable(s) is/are actually contributing to explaining Y.

NOTE: Just a simple explanation

$\endgroup$
-2
$\begingroup$

R-Squared tells us the variance of the dependent variable which is represented by the group of independent variables.

Correlation coefficient tells us how two variables move or interact with each other.

$\endgroup$
  • 6
    $\begingroup$ (-1) R-squared gives no information at all about the variance of the dependent variable (or even the residuals). $\endgroup$ – whuber Jan 25 at 16:38
  • 1
    $\begingroup$ I suspect that the poster knows that and simply left out the phrase proportion of in writing it down. $\endgroup$ – G. Grothendieck Jan 25 at 17:41
  • 3
    $\begingroup$ I think we often give people the benefit of the doubt for knowledge they might have-- when the simplest way to clarify their knowledge is to explicitly state the knowledge. From years of working with students, it was generally the pattern that those with a good grasp on the ideas explained them clearly with the proper terminology specific to statistics (i.e. "... proportion of the variation in Y is explained...", rather than failing to mention proportion or using variance instead of variation). Whuber is fair in pointing out the imprecision in the post, irrespective of why it exists. $\endgroup$ – LSC Jan 25 at 21:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy