8
$\begingroup$

Pearson correlation coefficient is calculated using the formula $r = \frac{cov(X,Y)}{\sqrt{var(X)} \sqrt{var(Y)}}$. How does this formula contain the information that the two variates $X$ and $Y$ are correlated or not? Or, how do we get this formula for the correlation coefficient?

$\endgroup$
7
$\begingroup$

What matter is $cov(X,Y)$. Denominator $\sqrt{var(X)var(Y)}$ is for getting rid of units of measure (if say $X$ is measured in meters and $Y$ in kilograms then $cov(X,Y)$ is measured in meter-kilograms which is hard to comprehend) and for standardization ($cor(X,Y)$ lies between -1 and 1 whatever variable values you have).

Now back to $cov(X,Y)$. This shows how variables vary together about their means, hence co-variance. Let us take an example.enter image description here

Lines are drawn at sample means $\bar X$ and $\bar Y$. The points in the upper right corner are where both $X_i$ and $Y_i$ are above their means and so both $(X_i-\bar X)$ and $(Y_i-\bar Y)$ are positive. The points in the lower left corner are below their means. In both cases product $(X_i-\bar X)(Y_i-\bar Y)$ is positive. On the contrary upper left and lower right are areas where this product is negative.

Now when computing covariance $cov(X,Y)=\frac1{n-1}\sum_{i=1}^n(X_i-\bar X)(Y_i-\bar Y)$ in this example points that give positive products $(X_i-\bar X)(Y_i-\bar Y)$ dominate, resulting positive covariance. This covariance is bigger when points are aligned closer to an imaginable line crossing the point $(\bar X,\bar Y)$.

As a last note, covariance shows only the strength of a linear relationship. If relationship is non linear, covariance is not able to detect it.

$\endgroup$
  • $\begingroup$ covariance shows only the strength of a linear relationship This is not true. Cov is sensitive to both strength of linearity and to magnitude of variation. Take X and Y, strictly linearly related. Then pull apart two extreme points in X, to enlarge var(X). The bivariate cloud is not linear anymore - it's only monotonic; still, cov(X,Y) became greater! However, if we now bring back the sum var(X)+var(Y) to its initial amount, cov(X,Y) will drop below, and below its initial value, reflecting the fact that we previously disturbed linearity. $\endgroup$ – ttnphns Jun 19 '12 at 7:51
  • $\begingroup$ Wow, that is interesting. $\endgroup$ – danas.zuokas Jun 19 '12 at 12:04
3
$\begingroup$

If, in the formula that you display, you remove the 'dividedness' of all three terms, cov(X,Y), var(X) and var(Y) by n-1, you get even more basic formula for r: $\frac{SCP(X,Y)}{\sqrt{SS(X)} \sqrt{SS(Y)}}$, where SCP is "sum cross-products" and SS is "sum of squares". Generally, this is the formula for cosine. But since X and Y are centered ("sum of cross-products of deviations" and "sum of squares of deviations") it becomes the formula for r, - r is the cosine between centered variables.

Now, cosine is the measure of proportionality; cos(X,Y)=1 when and only when Xi=kYi, that is when all points (i) lie on a straight line coming from the origin of the X vs Y coordinate system. If either the line doesn't come through the origin or points depart from the straight line cos will become smaller. Because Pearson r is the cos of the cloud which has been centered on both X and Y axes the line inevitably comes through the origin; and hence only departure of points from lying on the straight line can diminish r: r is the measure of linearity.

$\endgroup$
1
$\begingroup$

If r = 1, there is perfect linear correlation, if r = -1 there is perfect negative linear correlation, if r = 0, there is no linear correlation. The reason we divide by standard deviations of X and Y is to obtain a measure that does not depend on scale.

See this thread for more detailed answers.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.