I am trying to measure the correlation between two binary categorical variables. I tried in two ways.

  1. Use the chi-square test for independence of two-way table. The p-value as the correlation.

  2. Treat them as numerical variables with value of 0/1. Calculate the correlation between them.

The results are so different, the first gives 9.5e(-31) and the second gives around 0.3.

How to interpret these two correlations and why the difference can be so large?

  • 2
    $\begingroup$ Are you using the p-value to determine the strength of correlation? I don't think that's a good idea. $\endgroup$
    – Vishal
    Mar 30 '16 at 21:23
  • 3
    $\begingroup$ The second is a phi coefficient and is fine. But what makes you believe that the p-value from the chi-square test can be interpreted as a correlation? $\endgroup$
    – Wolfgang
    Mar 30 '16 at 21:23
  • $\begingroup$ @Vishal I think since the smaller P-value is, the less likely two variables are independent. That's very similar to a measure of strength of correlation. Not sure it makes sense. $\endgroup$
    – fangh
    Mar 31 '16 at 13:30
  • $\begingroup$ @Wolfgang That's my doubt here also. Can p-value of the A/B test here give the measure to strength of correlation. Why or why not? $\endgroup$
    – fangh
    Mar 31 '16 at 13:37
  • $\begingroup$ The p-value depends also upon sample size, and not only upon the correlation $\endgroup$ Sep 26 '18 at 13:46

The p-value from the chi-squared test cannot in any way be interpreted as a correlation. There are at least two reasons:

  • The p-value will depend on sample size, and not only on strength of association or correlation. With a given non-null association (measure of dependence) when the sample size grows, the p-value will tend to be lower.

  • When the correlation/association increases, the p-value will tend to decrease. So for a given sample size, association/correlation will be antimonotonic with p-value.

So to summarize: the p-value measures strength of evidence, not strength of association/correlation.

  • 1
    $\begingroup$ There is at least a third fundamental reason: As the p-value can never be negative it cannot -- unlike a correlation and several measures of association -- carry information on direction of relationship. $\endgroup$
    – Nick Cox
    Sep 27 '18 at 10:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.