Basic misunderstanding about calculating correlation coefficient I'm trying to build intuition and understand how the calculation of correlation coefficient (r) works. I'm reading the book "Statistics" by David Freedman et al, where they provide, in chapter 8, a series of steps to calculate r.
Essentially, the method provided in the book is:


*

*transform each variable (i.e., for both x and y) data points from raw to standard units

*multiply each data point's Zx by its Zy.

*average the products.

*Voilà. The average of products is the correlation coefficient r.



But something doesn't make sense to me. Let's say that hypothetically I have data that, after transformation to standard units, turns out to be:



*

*This means that the two variables x and y should have correlation of 1, because they perfectly match in their respective deviation from each variable's mean.


*

*But when apply the calculation method provided above, I get this bizarre result:

Average of Products $= \frac{9+4+1+0+9+4+1}{7} = 4$
Well, that's clearly wrong. We know that r ranges from -1 to 1, so what's wrong in my understanding?
 A: The fallacy here is that your hypothetical dataset is impossible. 
The consequence of standardizing is that each standardized variable has mean 0 and SD 1. You subtract the mean, so the new mean is 0, and you then divide by the SD, so the new SD is 1. 
Your hypothetical dataset has mean zero on each standardized variable, but its standard deviation on each standardized variable is much greater than 1, which is why your calculated correlation is outside the bounds possible. 
In other words, the recipe given by Freedman, Pisani and Purves -- which is just a variation on any other -- does depend on standardization having taken place. 
A: The problem with your hypothetical data is that it is not standardized. Both $z_x$ and $z_y$ have mean zero, but their standard deviation is not equal to one.
Why is this important? In the general case, your formula is calculating the sample covariance:
$\hat{cov}(x,y)= \frac{1}{n}\sum_{i=1}^n (x_i-\hat{\mu}_x)(y_i-\hat{\mu}_y)$
where $\hat{\mu}_x$ and $\hat{\mu}_y$ are the sample means (both equal to zero in your made-up data).
The sample correlation is:
$\hat{corr}(x,y)=\frac{\hat{cov}(x,y)}{\hat{\sigma}_x\hat{\sigma}_y}$
So your formula only gives the sample correlation if both standard deviations ($\hat{\sigma}_x$ and $\hat{\sigma}_y$) are equal to one, that is, if both variables are correctly standardized.
