I want to fully grasp the notion of $r^2$ describing the amount of variation between variables. Every web explanation is a bit mechanical and obtuse. I want to "get" the concept, not just mechanically use the numbers.
E.g.: Hours studied vs. test score
$r$ = .8
$r^2$ = .64
Start with the basic idea of variation. Your beginning model is the sum of the squared deviations from the mean. The R^2 value is the proportion of that variation that is accounted for by using an alternative model. For example, R-squared tells you how much of the variation in Y you can get rid of by summing up the squared distances from a regression line, rather than the mean.
I think this is made perfectly clear if we think about the simple regression problem plotted out. Consider a typical scatterplot where you have a predictor X along the horizontal axis and a response Y along the vertical axis.
The mean is a horizontal line on the plot where Y is constant. The total variation in Y is the sum of squared differences between the mean of Y and each individual data point. It's the distance between the mean line and every individual point squared and added up.
You can also calculate another measure of variability after you have the regression line from the model. This is the difference between each Y point and the regression line. Rather than each (Y - the mean) squared we get (Y - the point on the regression line) squared.
If the regression line is anything but horizontal, we're going to get less total distance when we use this fitted regression line rather than the mean--that is there is less unexplained variation. The ratio between the extra variation explained and the original variation is your R^2. It's the proportion of the original variation in your response that is explained by fitting that regression line.
Here is some R code for a graph with the mean, the regression line, and segments from the regression line to each point to help visualize:
library(ggplot2)
data(faithful)
plotdata <- aggregate( eruptions ~ waiting , data = faithful, FUN = mean)
linefit1 <- lm(eruptions ~ waiting, data = plotdata)
plotdata$expected <- predict(linefit1)
plotdata$sign <- residuals(linefit1) > 0
p <- ggplot(plotdata, aes(y=eruptions, x=waiting, xend=waiting, yend=expected) )
p + geom_point(shape = 1, size = 3) +
geom_smooth(method=lm, se=FALSE) +
geom_segment(aes(y=eruptions, x=waiting, xend=waiting, yend=expected, colour = sign),
data = plotdata) +
theme(legend.position="none") +
geom_hline(yintercept = mean(plotdata$eruptions), size = 1)
A mathematical demonstration of the relationship between the two is here: Pearson's correlation and least squares regression analysis.
I am not sure if there is a geometric or any other intuition that can be offered apart from the math but if I can think of one I will update this answer.
Update: Geometric Intuition
Here is a geometric intuition I came up with. Suppose that you have two variables $x$ and $y$ which are mean centered. (Assuming mean centered lets us ignore the intercept which simplifies the geometrical intuition a bit.) Let us first consider the geometry of linear regression. In linear regression, we model $y$ as follows:
$y = x\ \beta + \epsilon$.
Consider the situation when we have two observations from the above data generating process given by the pairs ($y_1,y_2$) and ($x_1,x_2$). We can view them as vectors in two-dimensional space as shown in the figure below:
alt text http://a.imageshack.us/img202/669/linearregression1.png
Thus, in terms of the above geometry, our goal is to find a $\beta$ such that the vector $x\ \beta$ is the closest possible to the vector $y$. Note that different choices of $\beta$ scale $x$ appropriately. Let $\hat{\beta}$ be the value of $\beta$ that is our best possible approximation of $y$ and denote $\hat{y} = x\ \hat{\beta}$. Thus,
$y = \hat{y} + \hat{\epsilon}$
From a geometrical perspective we have three vectors. $y$, $\hat{y}$ and $\hat{\epsilon}$. A little thought suggests that we must choose $\hat{\beta}$ such that three vectors look like the one below: alt text http://a.imageshack.us/img19/9524/intuitionlinearregressi.png
In other words, we need to choose $\beta$ such that the angle between $x\ \beta$ and $\hat{\epsilon}$ is 900.
So, how much variation in $y$ have we explained with this projection of $y$ onto the vector $x$. Since the data is mean centered the variance in $y$ is equals ($y_1^2+y_2^2$) which is the square of the distance between the point represented by the point $y$ and the origin. The variation in $\hat{y}$ is similarly the distance from the point $\hat{y}$ and the origin and so on.
By the Pythagorean theorem, we have:
$y^2 = \hat{y}^2 + \hat{\epsilon}^2$
Therefore, the proportion of the variance explained by $x$ is $\frac{\hat{y}^2}{y^2}$. Notice also that $cos(\theta) = \frac{\hat{y}}{y}$. and the wiki tells us that the geometrical interpretation of correlation is that correlation equals the cosine of the angle between the mean-centered vectors.
Therefore, we have the required relationship:
(Correlation)2 = Proportion of variation in $y$ explained by $x$.
Hope that helps.
The Regression By Eye applet could be of use if you're trying to develop some intuition.
It lets you generate data then guess a value for R, which you can then compare with the actual value.
