1
$\begingroup$

I am trying to measure the correlation between two binary categorical variables. I tried in two ways.

  1. Use the chi-square test for independence of two-way table. The p-value as the correlation.

  2. Treat them as numerical variables with value of 0/1. Calculate the correlation between them.

The results are so different, the first gives 9.5e(-31) and the second gives around 0.3.

How to interpret these two correlations and why the difference can be so large?

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ Are you using the p-value to determine the strength of correlation? I don't think that's a good idea. $\endgroup$
    – Vishal
    Commented Mar 30, 2016 at 21:23
  • 3
    $\begingroup$ The second is a phi coefficient and is fine. But what makes you believe that the p-value from the chi-square test can be interpreted as a correlation? $\endgroup$
    – Wolfgang
    Commented Mar 30, 2016 at 21:23
  • $\begingroup$ @Vishal I think since the smaller P-value is, the less likely two variables are independent. That's very similar to a measure of strength of correlation. Not sure it makes sense. $\endgroup$
    – fangh
    Commented Mar 31, 2016 at 13:30
  • $\begingroup$ @Wolfgang That's my doubt here also. Can p-value of the A/B test here give the measure to strength of correlation. Why or why not? $\endgroup$
    – fangh
    Commented Mar 31, 2016 at 13:37
  • $\begingroup$ The p-value depends also upon sample size, and not only upon the correlation $\endgroup$
    – kjetil b halvorsen ♦
    Commented Sep 26, 2018 at 13:46

1 Answer 1

Reset to default
1
$\begingroup$

The p-value from the chi-squared test cannot in any way be interpreted as a correlation. There are at least two reasons:

  • The p-value will depend on sample size, and not only on strength of association or correlation. With a given non-null association (measure of dependence) when the sample size grows, the p-value will tend to be lower.

  • When the correlation/association increases, the p-value will tend to decrease. So for a given sample size, association/correlation will be antimonotonic with p-value.

So to summarize: the p-value measures strength of evidence, not strength of association/correlation.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ There is at least a third fundamental reason: As the p-value can never be negative it cannot -- unlike a correlation and several measures of association -- carry information on direction of relationship. $\endgroup$
    – Nick Cox
    Commented Sep 27, 2018 at 10:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.