# r-squared: I have never fully grasped the interpretation

I want to fully grasp the notion of $r^2$ describing the amount of variation between variables. Every web explanation is a bit mechanical and obtuse. I want to "get" the concept, not just mechanically use the numbers.

E.g.: Hours studied vs. test score

$r$ = .8

$r^2$ = .64

• So, what does this mean?
• 64% of the variability of test scores can be explained by hours?
• How do we know that just by squaring?
-
 your question is not about R vs R-square (you understand that $0.8^2=0.64$) it is about interpretation of $r^2$. Please reformulate the title. – robin girard Aug 9 '10 at 15:03 a similar question: stats.stackexchange.com/questions/28139/… – Abe Apr 22 at 19:04

A mathematical demonstration of the relationship between the two is here: Pearson's correlation and least squares regression analysis.

I am not sure if there is a geometric or any other intuition that can be offered apart from the math but if I can think of one I will update this answer.

Update: Geometric Intuition

Here is a geometric intuition I came up with. Suppose that you have two variables $x$ and $y$ which are mean centered. (Assuming mean centered lets us ignore the intercept which simplifies the geometrical intuition a bit.) Let us first consider the geometry of linear regression. In linear regression, we model $y$ as follows:

$y = x\ \beta + \epsilon$.

Consider the situation when we have two observations from the above data generating process given by the pairs ($y_1,y_2$) and ($x_1,x_2$). We can view them as vectors in two-dimensional space as shown in the figure below:

Thus, in terms of the above geometry, our goal is to find a $\beta$ such that the vector $x\ \beta$ is the closest possible to the vector $y$. Note that different choices of $\beta$ scale $x$ appropriately. Let $\hat{\beta}$ be the value of $\beta$ that is our best possible approximation of $y$ and denote $\hat{y} = x\ \hat{\beta}$. Thus,

$y = \hat{y} + \hat{\epsilon}$

From a geometrical perspective we have three vectors. $y$, $\hat{y}$ and $\hat{\epsilon}$. A little thought suggests that we must choose $\hat{\beta}$ such that three vectors look like the one below:

In other words, we need to choose $\beta$ such that the angle between $x\ \beta$ and $\hat{\epsilon}$ is 900.

So, how much variation in $y$ have we explained with this projection of $y$ onto the vector $x$. Since the data is mean centered the variance in $y$ is equals ($y_1^2+y_2^2$) which is the square of the distance between the point represented by the point $y$ and the origin. The variation in $\hat{y}$ is similarly the distance from the point $\hat{y}$ and the origin and so on.

By the Pythagorean theorem, we have:

$y^2 = \hat{y}^2 + \hat{\epsilon}^2$

Therefore, the proportion of the variance explained by $x$ is $\frac{\hat{y}^2}{y^2}$. Notice also that $cos(\theta) = \frac{\hat{y}}{y}$. and the wiki tells us that the geometrical interpretation of correlation is that correlation equals the cosine of the angle between the mean-centered vectors.

Therefore, we have the required relationship:

(Correlation)2 = Proportion of variation in $y$ explained by $x$.

Hope that helps.

-
 I do appreciate your attempt at helping, but unfortunately, this just made things 10x worse. Are you really introducing trigonometry to explain r^2? You're way too smart to be a good teacher! – JackOfAll Aug 10 '10 at 1:49 I thought that you wanted to know why correlation^2 = R^2. In any case, different ways of understanding the same concept helps or at least that is my perspective. – user28 Aug 10 '10 at 8:47

Start with the basic idea of variation. Your beginning model is the sum of the squared deviations from the mean. The R^2 value is the proportion of that variation that is accounted for by using an alternative model. For example, R-squared tells you how much of the variation in Y you can get rid of by summing up the squared distances from a regression line, rather than the mean.

I think this is made perfectly clear if we consider that regression problem and imagine that plotted out. Imagine a typical scatterplot where you have a predictor X along the horizontal axis and a response Y along the vertical axis.

The mean is a horizontal line on the plot where Y is constant. The Y variation is the sum of squared differences between the mean of Y and each individual data point. It's the distance between the mean line and every individual point squared and added up.

You can also calculate another measure of variability after you have the regression line from the model. This is the difference between each Y point and the regression line. Rather than each (Y - the mean) squared we get (Y - the point on the regression line) squared.

If the regression line is anything but horizontal, we're going to get less total distance when we use that instead of the mean--that is there is less unexplained variation. The ratio between the extra variation explained and the original variation is your R^2. It's the proportion of the original variation in your response that is explained by fitting that regression line.

See the (quick and dirty) image below...

Here is some R code for a basic graph with mean and the regression line plotted to help visualize, but without some helpful notes:

data(trees)
plot((trees$Volume~trees$Girth))
abline(lm(trees$Volume~trees$Girth))
abline(lm(trees\$Volume~1))

-
 > The ratio between the variation explained and the original variation is your R^2 Let's see if I got this. If the original variation from mean totals 100, and the regression variation totals 20, then the ratio = 20/100 = .2 You're saying R^2 = .2 b/c 20% of the mean variation (red) is accounted for by the explained variation (green) (In the case of r=1) If the original variation totals 50, and the regression variation totals 0, then the ratio = 0/50 = 0 = 0% of the variation from the mean (red) is accounted for by the explained variation (green) I'd expect R^2 to be 1, not 0. – JackOfAll Aug 10 '10 at 1:51 R^2 = 1-(SSR/SST) or (SST-SSR)/SST. So, in your examples, R^2=.80 and 1.00. The difference between the regression line and each point is that left UNexplained by the fit. The rest is the proportion explained. Otherwise, that's exactly right. – Brett Magill Aug 10 '10 at 4:17 I edited that last paragraph to try to make it a bit clearer. Conceptually (and computationally) all you need is there. It might be clearer to actually add the formula and refer to the SST SSE and SSR, but then I was trying to get at it conceptually – Brett Magill Aug 10 '10 at 4:18 ie: R^2 is the proportion of the total variation from mean (SST) that is the difference b/w the expected regression value and mean value (SSE). In my example of hours vs. score, the regression value would be the expected test score based on correlation with hours studied. Any additional variation from that is attributed to SSR. For a given point, hours studied variable/regression explained x% of the total variation from the mean (SST). With a high r-value, "explained" is big percentage of SST compared to SSR. With a low r-value, "explained" is a lower percentage of SST compared to SSR. – JackOfAll Aug 10 '10 at 13:56

The Regression By Eye applet could be of use if you're trying to develop some intuition.

It lets you generate data then guess a value for R, which you can then compare with the actual value.

-