Can anyone help me understand the Pearson correlation formula? the sample $r$ = the mean of the products of the standard scores of variables $X$ and $Y$.

I kind of understand why they need to standardize $X$ and $Y$, but how to understand the products of both the z scores?

This formula is also called "product-moment correlation coefficient", but what's the rationale for the product action? I am not sure if I have made my question clear, but I just want to remember the formula intuitively.

  • 11
    You might want to read the paper "Thirteen Ways to Look at the Correlation Coefficient" (Rodgers & Nicewander 1988). As the title implies, it discusses thirteen different intuitive views of the correlation coefficient. So hopefully at least one will click :) – half-pass Sep 25 '13 at 1:53
  • 10
    13 Ways can be found here – Dimitriy V. Masterov Sep 25 '13 at 2:18
  • 4
    A 14th way to understand the correlation (in terms of products of the z scores) comes down to understanding the covariance of the standardized variables, as illustrated at stats.stackexchange.com/questions/18058/…. – whuber Sep 25 '13 at 15:06
  • 4
    ...And a 15th way uses the circles shown at stats.stackexchange.com/a/46508/919: a least-squares fit minimizes the total area of the circles (there are at least two ways to do this when the points do not precisely line up) and the correlation coefficient is then their average area (when both variables are standardized). – whuber Sep 25 '13 at 19:33
  • 2
    Possible duplicate of What is covariance in plain language? – kjetil b halvorsen Nov 3 '17 at 12:56

In the comments, 15 ways to understand the correlation coefficent were suggested:


The 13 ways discussed in the Rodgers and Nicewander article (The American Statistician, February 1988) are

  1. A Function of Raw Scores and Means,

    $$r =\frac{\sum\left(X_i - \bar{X}\right)\left(Y_i - \bar{Y}\right)}{\sqrt{\sum\left(X_i-\bar{X}\right)^2\left(Y_i-\bar{Y}\right)^2}}.$$

  2. Standardized Covariance,

    $$r = s_{XY}/(s_Xs_Y)$$

    where $s_{XY}$ is sample covariance and $s_X$ and $s_Y$ are sample standard deviations.

  3. Standardized Slope of the Regression Line,

    $$r = b_{Y\cdot X}\frac{s_X}{s_Y} = b_{X\cdot Y}\frac{s_Y}{s_X},$$

    where $b_{Y\cdot X}$ and $b_{X \cdot Y}$ are the slopes of the regression lines.

  4. The Geometric Mean of the Two Regression Slopes,

    $$r = \pm \sqrt{b_{Y\cdot X}b_{X\cdot Y}}.$$

  5. The Square Root of the Ratio of Two Variances (Proportion of Variability Accounted For),

    $$r = \sqrt{\frac{\sum\left(Y_i - \hat{Y_i}\right)^2}{\sum\left(Y_i-\bar{Y}\right)^2}} = \sqrt{\frac{SS_{REG}}{SS_{TOT}}} = \frac{s_\hat{Y}}{s_Y}.$$

  6. The Mean Cross-Product of Standardized Variables,

    $$r = \sum z_X z_Y / N.$$

  7. A Function of the Angle Between the Two Standardized Regression Lines. The two regression lines (of $Y$ vs. $X$ and $X$ vs. $Y$) are symmetric about the diagonal. Let the angle between the two lines be $\beta$. Then

    $$r = \sec(\beta)\pm \tan(\beta).$$

  8. A Function of the Angle Between the Two Variable Vectors,

    $$r = \cos(\alpha).$$

  9. A Rescaled Variance of the Difference Between Standardized Scores. Letting $z_Y - z_X$ be the difference between standardized $X$ and $Y$ variables for each observation,

    $$r = 1 - s^2_{(z_Y - z_X)} / 2 = s^2_{(z_Y+z_X)}/2 - 1.$$

  10. Estimated from the "Balloon" Rule,

    $$r \approx \sqrt{1 - (h/H)^2}$$

    where $H$ is the vertical range of the entire $X-Y$ scatterplot and $h$ is the range through the "center of the distribution on the $X$ axis" (that is, through the point of means).

  11. In Relation to the Bivariate Ellipses of Isoconcentration,

    $$r = \frac{D^2 - d^2}{D^2 + d^2}$$

    where $D$ and $d$ are the major and minor axis lengths, respectively. $r$ also equals the slope of the tangent line of an isocontour (in standardized coordinates) at the point the contour crosses the vertical axis.

  12. A Function of Test Statistics from Designed Experiments,

    $$r = \frac{t}{\sqrt{t^2 + n-2}}$$

    where $t$ is the test statistic in a two-independent sample $t$ test for a designed experiment with two treatment conditions (coded as $X=0, 1$) and $n$ is the combined total number of observations in the two treatment groups.

  13. The Ratio of Two Means. Assume bivariate normality and standardize the variables. Select some arbitrarily large value $X_c$ of $X$. Then

    $$r = \frac{\mathbb{E}(Y\,|\,X\gt X_c)}{\mathbb{E}(X\,|\,X\gt X_c)}.$$

(Most of this is verbatim, with very slight changes in some of the notation.)

Some other methods (perhaps original to this site) are

  • Via circles. $r$ is the slope of the regression line in standardized coordinates. This line can be characterized in various ways, including geometric ones, such as minimizing the total area of circles drawn between the line and the data points in a scatterplot.

  • By coloring rectangles. Covariance can be assessed by coloring rectangles in a scatterplot (that is, by summing signed areas of rectangles). When the scatterplot is standardized, the net amount of color--the total signed error--is $r$.

  • 1
    Thank you, @Avraham, for trying to bring this unanswered thread to some closure by posting an answer here. – whuber Jun 24 '14 at 16:24

Your Answer

 
discard

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.