# Truncating data reduces correlation?

Here is an argument I came across: by limiting your sample by some range of one of the variables the (Pearson) correlation coefficient between the two variables is likely to be reduced.

I can't see the logic. I would think it could both increase or reduce, depending on whether the linear relationship is stronger or not for that interval at hand.

Any clues?

• However, "is likely to" does not imply "will always." – Penguin_Knight Mar 19 '15 at 16:19
• Of course! But it implies more likely than not, which I have hard time accepting. – snoram Mar 19 '15 at 16:20
• "Limiting your sample" sounds like truncation. Censoring means that values in the sample are replaced by censoring limits. The effect on the nominal correlation coefficient (as incorrectly calculated by replacing each censored value by its censoring limit) is profoundly different from removing such values altogether (truncation). Which one do you really mean? – whuber Mar 19 '15 at 16:34
• I mean "removing such values altogether (truncation)". Thanks, I will change the title. – snoram Mar 19 '15 at 16:36
• This phenomenon is illustrated in my answer at stats.stackexchange.com/a/13317/919, which slices one variable into quantiles and quotes the $R^2$ (squared correlation coefficient) for the data within each slice. – whuber Mar 19 '15 at 21:41

Imagine for a moment we're looking at a regression problem. The squared correlation between the two variables ($r^2$) is $R^2$, the coefficient of determination, which is $1-\frac{s^2_\epsilon}{\text{Var}(y)}$. When you restrict the range of $x$, you also reduce the range of $y$, so $\text{Var}(y)$ goes down with it, while $s^2_\epsilon$ (the noise about the line) should hardly change, since it still has an expected value of $\sigma^2_\epsilon$. Here's an example of that: Since the denominator of the fraction decreases while the numerator hardly changes, the fraction gets larger, so $R^2$ gets smaller, so $r^2(x,y)$ and hence $|r|$ will be smaller. So we really should expect that the size of the correlation decreases.