6
$\begingroup$

As the title to my question says, I am confused as to when the $R^2$ of a model fit does not equal the slope of the regression between observed and predicted values.

I am trying to present model prediction statistics in a similar way to those presented in the summary figures of the Globcolor validation report (link) - (e.g. figure from page 53 of the .pdf):

enter image description here

Here we see that they present the plot of observed versus predicted Chlorophyll concentrations, as well as statistics relating to its regression (e.g. the dashed line: $R^2$, $RMS$, $\alpha$ - intercept, and $\beta$ - slope).

My issue is that in my comparisons, I always get exactly the same value for the overall model fit $R^2$ and $\beta$-slope of the observed versus predicted regression.

Basic question: When (if ever) can these be different?

I have included a basic example of my problem in the following R script:

set.seed(1)
n <- 100
x <- runif(n)
e <- rnorm(n)
a <- 3
b <- 5
y <- a + x*b + e

#fit model
fit <- lm( y ~ x )

#plot regression
plot(x,y)
abline(fit)

#plot predicted versus observed
png("plot.png", units="in", width=5, height=5, res=400)
par(mar=c(5,5,1,1))
pred <- predict(fit)
plot(y, pred, xlim=range(c(y,pred)), ylim=range(c(y,pred)), xlab="observed", ylab="predicted")
abline(0,1, lwd=2, col=8)

#add regression
fit2 <- lm(pred ~ y)
lgd <- c(
    paste("R^2 =", round(summary(fit2)$r.squared,3)),
    paste("Offset =", round(coef(fit2)[1],3)),
    paste("Slope =", round(coef(fit2)[2],3))
)
legend("topleft", legend=lgd)
abline(fit2, lwd=2)
legend("bottomright", legend=c("predicted ~ observed", "1:1"), col=c(1,8), lty=1, lwd=2)

dev.off()

cor(pred, y)^2 # also the same

enter image description here

$\endgroup$
5
$\begingroup$

I always get exactly the same value for the overall model fit $R^2$ and slope of the observed versus predicted regression.

This will be true provided a constant term is included in the overall model. Why?

  1. $R^2$ measures the variance of the fit $\hat Y$ relative to the variance of $Y$ (provided the model includes a constant).

  2. Regressing $\hat Y$ against $Y$ or $Y$ against $\hat Y$ must produce identical standardized slopes $\hat\beta_{\hat{Y}Y} = \hat\beta_{Y\hat{Y}}$. This is because the standardized slope in a univariate regression of $Y$ against any $X$ is their correlation coefficient $\rho_{XY}$, which is symmetric in $X$ and $Y$.

  3. The standardized slope $\hat \beta_{XY}$ in any univariate regression of $Y$ against any $X$ is related to the slope $\hat b_{XY}$ via

    $$\hat \beta_{XY} = \hat b_{XY} \frac{\text{SD}(X)}{\text{SD}(Y)}.$$

  4. Regressing $Y$ against $\hat Y$ must have a unit slope $\hat b_{\hat{Y}Y}$. Geometrically, $\hat Y$ is the projection of $Y$ onto the column space of the design matrix and the regression of $Y$ against $\hat Y$ is $1$ times the component of $Y$ on that projection.

Putting these all together (in order) yields

$$R^2 = \rho^2_{\hat{Y}Y} = \hat\beta_{\hat{Y}Y}\hat\beta_{Y\hat{Y}} = \left(\hat b_{Y\hat{Y}} \frac{\text{SD}(Y)}{\text{SD}(\hat{Y})}\right)\left(\hat b_{\hat{Y}Y} \frac{\text{SD}(\hat{Y})}{\text{SD}(Y)}\right) = \hat b_{Y\hat{Y}}\hat b_{\hat{Y}Y} = \hat b_{Y\hat{Y}},$$

QED.

The result is not necessarily true when the model does not include a constant: just about any random simulation, as shown below, will give a counterexample.

n <- 10; d <- 2
x <- matrix(rnorm(n*d), ncol=d)
y <- x %*% (1:d) + rnorm(n, 3)
fit <- lm(y ~ x)
y.hat <- predict(fit)
#
# Look for the appearances of R^2 in the output.
#
var(y.hat) / var(y) # R^2
with(summary(lm(y.hat ~ y)), c(coefficients["y", 1], r.squared))
with(summary(lm(y ~ y.hat)), c(coefficients["y.hat", 1], r.squared))
#
# Repeat without a constant term: the same consistency among
# the output occurs, *but the slopes are not equal to R^2*.
#
with(summary(lm(y.hat ~ y - 1)), c(coefficients["y", 1], r.squared))
with(summary(lm(y ~ y.hat - 1)), c(coefficients["y.hat", 1], r.squared))
$\endgroup$
  • $\begingroup$ Seeing proofs like this sometimes makes me wish I had studied statistics in university. :) It looks so easy, but would have taken me hours. $\endgroup$ – Roland Feb 26 '14 at 16:15
  • 2
    $\begingroup$ @Roland Believe it or not, I learned every one of these facts by teaching a beginning statistics class at the university level. It's all in books like Statistics by Freedman, Pisani, & Purves. (Consult any edition.) I am not trying to imply that graduates of this class would be expected to come up with (1) through (4) and put them together to solve this problem, but it is of interest that all the pieces of the puzzle are completely elementary (and, with a bit of practice in such thinking, somewhat intuitive). $\endgroup$ – whuber Feb 26 '14 at 16:19
  • $\begingroup$ @whuber - Really great answer as usual. Thanks a lot for taking the time to explain this to me. I'm going to stop banging my head against the wall now... $\endgroup$ – Marc in the box Feb 26 '14 at 16:23
  • $\begingroup$ btw - I think this particular ocean color algorithm (i.e. model) uses a summation of reflectance spectra signatures (e.g. baseline water + chlorophyll) to estimate chl concentrations; thus, it's very possible that there is not an intercept. I'll have to look into that. Cheers and thanks! $\endgroup$ – Marc in the box Feb 26 '14 at 16:28
  • $\begingroup$ Since it appears to be a log-log model, the intercept would correspond to a multiplicative term on the original scale--and this would appear to be essential if only to make units of measurement commensurate (between predicted values, which are combinations of spectral intensities, and actual values, which are concentrations). $\endgroup$ – whuber Feb 26 '14 at 16:33
0
$\begingroup$

$R^2$ doesn't have to be equal to $\beta$. you either have a rare coincidence, or reading the same field from the fit object somehow.

$\endgroup$
  • 2
    $\begingroup$ You do not seem to be addressing the question concerning $R^2$ of the regression of predicted against observed values. $\endgroup$ – whuber Feb 26 '14 at 14:40
  • $\begingroup$ right, missed that $\endgroup$ – Aksakal Feb 26 '14 at 14:45
  • $\begingroup$ I don't think this is a coincidence. all.equal(cor(y,x)^2, unname(coef(lm(predict(fit)~y))[2])) is TRUE for different parameters and random seeds. I believe if someone sits down and does the maths this could be proven analytically. $\endgroup$ – Roland Feb 26 '14 at 14:49
  • $\begingroup$ @Roland - Yes, this seems to be the case for every model fit that I have done. So, perhaps the error is in the Globcolour analyses - How they calculated their summary statistics is not well documented... $\endgroup$ – Marc in the box Feb 26 '14 at 15:03
  • $\begingroup$ actually, the graph from the Globcolour report doesn't seem to be of predicted vs. observed. i think they're regressing some data on another set, which they call "in-situ" $\endgroup$ – Aksakal Feb 26 '14 at 15:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.