Symmetry of standardized regression coefficient (=Pearson correlation) in linear regression

Question

Suppose 2 continuous variables X and Y. Their Pearson correlation equals 0.8. This correlation is symmetric (it does not assume a dependent or independent variable). We proceed to a linear regression, in which we regress Y on X. Regardless of the unstandardized solution, we immediately go to the standardized solution (both Y and X standardized), leading to the following solution:

Z_Y = 0.8*Z_X

I interpret this as: if X increases with 1 standard deviation, Y increases with 0.8 standard deviations. Now, suppose, for whatever reason, we also regress X on Y. That standardized solution is:

Z_X = 0.8*Z_Y

This true because the standardized regression coefficient is equal to the correlation coefficient, and thus both are symmetric. When interpreting this result, I would say: if Y increases with 1 standard deviation, X increases with 0.8 standard deviations.

As both interpretations seem contradictory, how can they be reconciled? I know the correlation coefficient (or standardized regression coefficient) cannot go below -1 or above 1, but why is the second equation not: Z_X = (1/0.8)*Z_Y?

I cannot get my head around this, so clearly I am missing out on something fundamental here. Many thanks!

Answer 1

You seem to be seeing something like $y = 0.8x\iff x = 0.8 y$ and thinking that looks ridiculous. Yes, that is ridiculous, and the right relationship is $y = 0.8x\iff x = \frac{y}{0.8}$, as you have noticed the algebra says.

However, those are not the regression equations! The equations below are the regression equations.

$$ \mathbb E[y\vert x] = 0.8 x\\ \Big\Updownarrow\\ \mathbb E[x\vert y] = 0.8 y\\ $$

Therefore, the algebraic rearrangement that leads to the apparent contradiction is not the correct algebraic rearrangement.

Answer 2

Thanks for this answer. In the meantime, I also got to understand the issue better.

As Dave shows in the regression formula, we are modeling the "expected value of the dependent variable, given fixed covariate values", i.e. in regression we model the conditional mean. The mistake I made was to think that when the independent variable increases with 1 sd, that the dependent variable decreased with 0.8 sd. More correct is to say that the conditional mean of the dependent variable decreases with 0.8 sd. This directly relates to the concept of "regression to the mean", which says that: if you take an extreme group on one variable, the average of this group on another variable will always be closer to the mean (unless there is perfect correlation). So, if X increases with 1 sd, then the conditional mean of Y decreases with 0.8 sd, and if Y increases with 1 sd, then the conditional mean of X decreases with 0.8 sd. Both statements do not contradict each other. In the former statement, you move up in the distribution of X and model the mean of Y, in the latter statement, you move up in the distribution of Y and model the mean of X.