# Counter-intuitive correlations vs. observed proportions

I'm confronting something pretty weird. I've got a household-level dataset, and I want to calculate the correlation between a binary variable A (taking values yes (2) or no (1)) and whether the household is urban or rural (a dummy taking values 0 or 1).

I'm using Stata and the correlation I get is positive and significant at 95%. My interpretation: urban households are more likely to say yes.

However, when I just look at the relative proportions of variable A for rural households and urban households, I see that urban households actually say yes less often.

I'm completely baffled. If you're observing a positive correlation between two dummy variables, shouldn't the observed relative proportions naturally follow that as well?

I've had this happen for other variables in a completely different dataset as well. I've double- and triple-checked my Stata code, just to make sure that there's not some stupid mistake along the way, but it all seems kosher. Is this possible? What interpretation do I give it?

-
Why have you coded the $A$ variable as $1/2$? The picture would be clearer if it was also $0/1$. – Alecos Papadopoulos Aug 18 '14 at 14:46
Do I take it right that you are using the dummies as if they were continuous variables? – James Aug 18 '14 at 14:47

These results are contradictory regarding what they imply in terms of true underlying probabilities, so either something is wrong (with the code, with the data, with the statistical test...), or, if everything is fine technically, they reveal an interesting situation (regarding our tools and our methodology, not the real-world phenomenon).

I code the $A$ variable as $0/1$ binary ($1$ = yes), and denote the household status as $X$ ($1$= urban).

The sign of the sample correlation coefficient depends on the sign of the sample covariance,

$$\operatorname{\hat Cov}(A,X) = \frac 1n\sum_{i=1}^na_ix_i - \bar a\cdot \bar x$$

the bar indicating the sample mean, and $n$ being the full sample size. Since both variables are $0/1$ binary, then one can realize that

$$\frac 1n\sum_{i=1}^na_ix_i = \hat P(A=1, X=1)$$

and

$$\bar a = \hat P(A=1),\;\;\; \bar x = \hat P(X=1)$$

The positive correlation coefficient result you found, if it indeed estimates well the true probabilities (so I dispense with the hat) implies therefore

$$P(A=1, X=1) > P(A=1)\cdot P(X=1) \tag{1}$$

Let's turn to the second result. I guess you calculated sub-sample proportions, and sub-sample proportions estimate conditional probabilities. So this result, expressed in probability terms, is

$$P(A=1 \mid X=0) > P(A=1 \mid X=1) \tag {2}$$

Using standard rules, $(2)$ can be written

$$\frac {P(A=1, X=0)}{P(X=0)} > \frac {P(A=1, X=1)}{P(X=1)} \tag {2a}$$

which combined with $(1)$ leads us to

$$P(A=1)\cdot P(X=1) < P(A=1, X=1) < \frac {P(A=1, X=0)P(X=1)}{P(X=0)}$$

$$\Rightarrow P(A=1)P(X=0) < P(A=1, X=0) \tag{3}$$

The right-hand side of $(3)$ can be written $$P(A=1, X=0) = P(A=1) - P(A=1,X=1)$$

Inserting in $(3)$ we have

$$P(A=1)P(X=0) < P(A=1) - P(A=1,X=1)$$

Rearranging

$$P(A=1,X=1) < P(A=1)\cdot [1-P(X=0)]$$

$$\Rightarrow P(A=1,X=1) < P(A=1)P(X=1) \tag{4}$$

and we have been led to a contradiction (compare $(4)$ with $(1)$). So your two empirical results cannot in reality hold together.

"But I 've found them!" you can object.
"No, you didn't", I will reply. What you found from your data (related say to the positive correlation), is a numerical estimate, which you then tested statistically in order to decide whether you will accept it as "true" (i.e. as holding also in reality). And in order to execute the test you used something that is totally unrelated to the data -the significance level of the test. It bears no weight that you chose an "accepted" significance level -it is still an arbitrary significance level.

Apart from recoding your $A$ variable as $0/1$ (so that things are clear), and maybe check once more all technical aspects, you can explore the following aspects:

a) For what significance level do the positive correlation becomes "statistically insignificant" and

b) What are the confidence intervals around the two conditional probabilities you calculated (the sub-sample proportions of saying "yes"). Do they overlap; Could you test for whether they can be considered statistically different from one another?

I suspect that the most reasonable conclusion here will be that the data set cannot really provide evidence on whether the variable "$A$" depends or not on the household being urban or rural.

-
Thanks, Alecos. I'm embarrassed to admit that, on quadruple-checking, I was misreading the correlations and simply got the sign wrong. Mea facepalm. Reassuring to know that it should be theoretically impossible - if I read your post correctly (it's been a long time since I've studied stats formally). Though it makes me wonder about this "common misconception" re: correlation and linearity: en.wikipedia.org/wiki/… – goblin-esque Aug 20 '14 at 6:18
Well, you just joined the ranks of great mathematicians that for a while thought they have proved a great unsolved conjecture, only to realize that they had got a sign wrong somewhere! As for correlation and linearity, indeed, it is not correct to think that "non-zero correlation means a linear relationship". $\pm 1$ correlation does mean a "clean" linear relationship, but everything in between, can reflect any relationship. Maybe this post will also help, stats.stackexchange.com/questions/107929/… – Alecos Papadopoulos Aug 20 '14 at 7:19