Pearson correlation coefficient is calculated using the formula $r = \frac{cov(X,Y)}{\sqrt{var(X)} \sqrt{var(Y)}}$. How does this formula contain the information that the two variates $X$ and $Y$ are correlated or not? Or, how do we get this formula for the correlation coefficient?
$\begingroup$
$\endgroup$
Add a comment
|
3 Answers
$\begingroup$
$\endgroup$
2
What matter is $cov(X,Y)$. Denominator $\sqrt{var(X)var(Y)}$ is for getting rid of units of measure (if say $X$ is measured in meters and $Y$ in kilograms then $cov(X,Y)$ is measured in meter-kilograms which is hard to comprehend) and for standardization ($cor(X,Y)$ lies between -1 and 1 whatever variable values you have).
Now back to $cov(X,Y)$. This shows how variables vary together about their means, hence co-variance. Let us take an example.
Lines are drawn at sample means $\bar X$ and $\bar Y$. The points in the upper right corner are where both $X_i$ and $Y_i$ are above their means and so both $(X_i-\bar X)$ and $(Y_i-\bar Y)$ are positive. The points in the lower left corner are below their means. In both cases product $(X_i-\bar X)(Y_i-\bar Y)$ is positive. On the contrary upper left and lower right are areas where this product is negative.
Now when computing covariance $cov(X,Y)=\frac1{n-1}\sum_{i=1}^n(X_i-\bar X)(Y_i-\bar Y)$ in this example points that give positive products $(X_i-\bar X)(Y_i-\bar Y)$ dominate, resulting positive covariance. This covariance is bigger when points are aligned closer to an imaginable line crossing the point $(\bar X,\bar Y)$.
As a last note, covariance shows only the strength of a linear relationship. If relationship is non linear, covariance is not able to detect it.
-
$\begingroup$
covariance shows only the strength of a linear relationshipThis is not true. Cov is sensitive to both strength of linearity and to magnitude of variation. Take X and Y, strictly linearly related. Then pull apart two extreme points in X, to enlarge var(X). The bivariate cloud is not linear anymore - it's only monotonic; still, cov(X,Y) became greater! However, if we now bring back the sum var(X)+var(Y) to its initial amount, cov(X,Y) will drop below, and below its initial value, reflecting the fact that we previously disturbed linearity. $\endgroup$– ttnphnsJun 19, 2012 at 7:51 -
$\begingroup$
$\endgroup$
If, in the formula that you display, you remove the 'dividedness' of all three terms, cov(X,Y), var(X) and var(Y) by n-1, you get even more basic formula for r: $\frac{SCP(X,Y)}{\sqrt{SS(X)} \sqrt{SS(Y)}}$, where SCP is "sum cross-products" and SS is "sum of squares". Generally, this is the formula for cosine. But since X and Y are centered ("sum of cross-products of deviations" and "sum of squares of deviations") it becomes the formula for r, - r is the cosine between centered variables.
Now, cosine is the measure of proportionality; cos(X,Y)=1 when and only when Xi=kYi, that is when all points (i) lie on a straight line coming from the origin of the X vs Y coordinate system. If either the line doesn't come through the origin or points depart from the straight line cos will become smaller. Because Pearson r is the cos of the cloud which has been centered on both X and Y axes the line inevitably comes through the origin; and hence only departure of points from lying on the straight line can diminish r: r is the measure of linearity.
$\begingroup$
$\endgroup$
If r = 1, there is perfect linear correlation, if r = -1 there is perfect negative linear correlation, if r = 0, there is no linear correlation. The reason we divide by standard deviations of X and Y is to obtain a measure that does not depend on scale.
See this thread for more detailed answers.
