Interpretation of the Pearson Correlation with respect to Z-Scores

Question

I'm TAing an introductory statistics course in a social sciences department, and we're getting questions about a line from the students' SPSS lab guide. It has to do with interpreting Pearson's r, and I've never quite seen it expressed this way:

"The correlation coefficient expresses relationship in terms of z scores (standard deviations). For instance, the correlation of −.609 conveys that as the independent variable X goes up by one standard deviation, the dependent variable Y is predicted to decrease by .609 standard deviations."

I'm wondering if this is

a) accurate (is r literally a z-score?)

and if

b) someone can flesh out what this is, in fact, trying to say.

It's not that I'm unfamiliar with Pearson's r - far from it, in fact. But I didn't really learn to interpret them in terms of z-scores. I'm aware that r can be calculated as the sum of the products of each pair of z-scores (for X and Y, that is) divided by the sample size, but I'm not quite sure what they're getting at. The bit I've quoted makes me think of the good ol' explanation of regression betas that you always get in intro stats, but I'm otherwise a bit stumped. (And embarrassed, too - seems so basic!)

Anyone want to take a crack at this?

Thanks!

l

Answer 1

The Pearson sample correlation coefficient can be written as:

$$r = \frac{1}{n-1} \sum_{i=1}^n z_{1,i} \cdot z_{2,i} \quad \quad \quad z_{k,i} = \frac{x_{k,i} - \bar{x}_k}{s_k}.$$

This result means that the sample correlation of two vectors $\boldsymbol{x}_1$ and $\boldsymbol{x}_2$ is equivalent to the sample correlation of their z-scores $\boldsymbol{z}_1$ and $\boldsymbol{z}_2$ (i.e., sample correlation is determined through the z-scores). Hence, it is accurate to say that the correlation coefficient expresses a relationship between the z-scores of the two sample vectors.

The second part of the statement, about the predictive effect of a change in one variable, is not true in general, but is true in the special case where the underlying data is jointly-normal (so long as we interpret the statement without conflating correlation and cause$^\dagger$). If we have an underlying normal distribution $(X_1, X_2) \sim \text{N}$ then the expected difference in $Z_2$ conditional on an "increase" from $Z_1 = x$ to $Z_1 = x+k$ is:

$$\text{Expected difference } (\Delta = k) = \mathbb{E}(Z_2 | Z_1 = x + k) - \mathbb{E}(Z_2 | Z_1 = x) = \rho \cdot k.$$

Hence, replacing the true correlation with the sample correlation you would have the predictive result:

$$\text{Predicted change}(\Delta = k) = \mathbb{E}(Z_2 | Z_1 = x + k) - \mathbb{E}(Z_2 | Z_1 = x) = r \cdot k.$$

Now, taking $k=1$ yields an interpretation of $r$ as the predictive change in this case:

$$\text{Predicted change}(\Delta = 1) = r \cdot 1 = r.$$

Hence, we see that for data from an underlying joint-normal distribution, an "increase" of one standard deviation for one of the variables, leads to a predictive change of $r$ standard deviations for the other variable. Note that this result is not a general result, and holds only in the case where the underlying distribution of the data is jointly normal. The second part of the statement should therefore be interpreted as a "rule of thumb" that applies in the jointly-normal case, but would apply only approximately for other distributions.

---

$^\dagger$ Note that in the above exposition we need to be careful not to conflate correlation with cause. Strictly speaking, if we increase $X_1$ through some action, then it is not appropriate to make a prediction based on the correlation, since we now need to know the causal effect of that increase. Hence, the above equations should be interpreted as predictive changes comparing two different observations of $X_1$ that differ by a specified amount. We have indicated this by referring to the "increase" in quotation marks.

Answer 2

Consider a simple linear regression on ($y_i$, $x_i$), $i=1,..,n: y_i = \alpha + \beta*x_i + e_i$, where $e_i$ is error (and the usual regression assumptions) and take a look at the regression coefficient: $\hat{\beta} = S_{xy}/S_{xx}$, which can be written as a function of the correlation coefficient r. Specificially, $\hat{\beta} = r \sigma_y/\sigma_x$. Now interpret the regression coefficient as a 1 unit change in x results in $\beta * 1$ unit change in $y$. Therefore, a 1 std. dev. ($\sigma_x$) change in $x$ results in $\hat{\beta} * 1 \sigma_x$ change in $y = r \sigma_y$ change in $y$.

Good luck with your teaching!