Can anyone help me understand the Pearson correlation formula? the sample $r$ = the mean of the products of the standard scores of variables $X$ and $Y$.
I kind of understand why they need to standardize $X$ and $Y$, but how to understand the products of both the z scores?
This formula is also called "product-moment correlation coefficient", but what's the rationale for the product action? I am not sure if I have made my question clear, but I just want to remember the formula intuitively.
In the comments, 15 ways to understand the correlation coefficent were suggested:
The 13 ways discussed in the Rodgers and Nicewander article (The American Statistician, February 1988) are
A Function of Raw Scores and Means,
$$r =\frac{\sum\left(X_i - \bar{X}\right)\left(Y_i - \bar{Y}\right)}{\sqrt{\sum\left(X_i-\bar{X}\right)^2\left(Y_i-\bar{Y}\right)^2}}.$$
Standardized Covariance,
$$r = s_{XY}/(s_Xs_Y)$$
where $s_{XY}$ is sample covariance and $s_X$ and $s_Y$ are sample standard deviations.
Standardized Slope of the Regression Line,
$$r = b_{Y\cdot X}\frac{s_X}{s_Y} = b_{X\cdot Y}\frac{s_Y}{s_X},$$
where $b_{Y\cdot X}$ and $b_{X \cdot Y}$ are the slopes of the regression lines.
The Geometric Mean of the Two Regression Slopes,
$$r = \pm \sqrt{b_{Y\cdot X}b_{X\cdot Y}}.$$
The Square Root of the Ratio of Two Variances (Proportion of Variability Accounted For),
$$r = \sqrt{\frac{\sum\left(Y_i - \hat{Y_i}\right)^2}{\sum\left(Y_i-\bar{Y}\right)^2}} = \sqrt{\frac{SS_{REG}}{SS_{TOT}}} = \frac{s_\hat{Y}}{s_Y}.$$
The Mean Cross-Product of Standardized Variables,
$$r = \sum z_X z_Y / N.$$
A Function of the Angle Between the Two Standardized Regression Lines. The two regression lines (of $Y$ vs. $X$ and $X$ vs. $Y$) are symmetric about the diagonal. Let the angle between the two lines be $\beta$. Then
$$r = \sec(\beta)\pm \tan(\beta).$$
A Function of the Angle Between the Two Variable Vectors,
$$r = \cos(\alpha).$$
A Rescaled Variance of the Difference Between Standardized Scores. Letting $z_Y - z_X$ be the difference between standardized $X$ and $Y$ variables for each observation,
$$r = 1 - s^2_{(z_Y - z_X)} / 2 = s^2_{(z_Y+z_X)}/2 - 1.$$
Estimated from the "Balloon" Rule,
$$r \approx \sqrt{1 - (h/H)^2}$$
where $H$ is the vertical range of the entire $X-Y$ scatterplot and $h$ is the range through the "center of the distribution on the $X$ axis" (that is, through the point of means).
In Relation to the Bivariate Ellipses of Isoconcentration,
$$r = \frac{D^2 - d^2}{D^2 + d^2}$$
where $D$ and $d$ are the major and minor axis lengths, respectively. $r$ also equals the slope of the tangent line of an isocontour (in standardized coordinates) at the point the contour crosses the vertical axis.
A Function of Test Statistics from Designed Experiments,
$$r = \frac{t}{\sqrt{t^2 + n-2}}$$
where $t$ is the test statistic in a two-independent sample $t$ test for a designed experiment with two treatment conditions (coded as $X=0, 1$) and $n$ is the combined total number of observations in the two treatment groups.
The Ratio of Two Means. Assume bivariate normality and standardize the variables. Select some arbitrarily large value $X_c$ of $X$. Then
$$r = \frac{\mathbb{E}(Y\,|\,X\gt X_c)}{\mathbb{E}(X\,|\,X\gt X_c)}.$$
(Most of this is verbatim, with very slight changes in some of the notation.)
Some other methods (perhaps original to this site) are
Via circles. $r$ is the slope of the regression line in standardized coordinates. This line can be characterized in various ways, including geometric ones, such as minimizing the total area of circles drawn between the line and the data points in a scatterplot.
By coloring rectangles. Covariance can be assessed by coloring rectangles in a scatterplot (that is, by summing signed areas of rectangles). When the scatterplot is standardized, the net amount of color--the total signed error--is $r$.
