I want to find the association between variables and Cramérs V works like a treat for matrices of sizes greater than 2$\times$2. However, for matrices with low frequencies, it does not work well. For the following contingency matrix, I get the result as 0.5. How can I account for the same?

  1 2  
a 2 0  
b 0 2  

Here is my code:

def cramers_stat(confusion_matrix):  
  chi2 = ss.chi2_contingency(confusion_matrix)[0]  
  n = confusion_matrix.sum().sum()  
  return np.sqrt(chi2 / (n*(min(confusion_matrix.shape)-1)))  

confusion_matrix is my input, in this case the matrix I mentioned above. I understand for good results, I need a matrix frequency above 5, but for perfect association as the case above I expected the result to be 1.

  • $\begingroup$ You should use the relevant tag for your language. That looks like python. Please also edit your question to make all instances of the pronoun "$I$" uppercase. $\endgroup$ – Glen_b -Reinstate Monica Jul 11 '17 at 4:21
  • 1
    $\begingroup$ Note that the requirement relating to 5 is not about the frequencies but about expected frequencies ... but it doesn't relate to computation of $V$, only to the approximate correctness of the significance levels in the chi-squared test (and then it's often too stringent) $\endgroup$ – Glen_b -Reinstate Monica Jul 11 '17 at 5:34
  • $\begingroup$ Thats something I was wondering as well, I guess it helps to get context before coding blindly and being too result oriented $\endgroup$ – avz2611 Jul 11 '17 at 5:37

For that table, Cramer's V should be 1.

From the relevant Wikipedia page, we can write $V$ in terms of the chi-squared value

$$V={\sqrt {{\frac {\chi ^{2}/n}{\min(k-1,r-1)}}}}$$

(this appears to be the formula you're attempting to implement).

Here all the expected values are 1 (row total x column total / overall total), and so each cell's $(O_i-E_i)^2/E_i$ is $1$.

So $\chi^2=4$. Further, $n=4$ and $k$ and $r$ are each $2$ so $\min(k-1,r-1)=1$. Immediately we have $V=\sqrt{(4/4)/1}=1$.

(You should have carried out such a computation yourself! How are you checking your code is correctly implemented if you're not checking simple cases like this by hand?)

So you did something wrong somewhere. It may be that your chi-squared value has been corrected, for example, but any number of other things may be wrong (you don't say what any of the component variables' values were).


Edit: It turns out that the supposition I made in that last sentence above is the case -- the python function that computes the chi-square applies Yates' continuity correction by default, which would reduce the statistic and hence make Cramér's $V$ less than $1$; you will want the uncorrected chi-square for that.

However, this doesn't necessarily account for the entire difference and you should check carefully the remaining parts of the calculation as well.

  • $\begingroup$ Hey, as i had used pre existing module of chi-square from python i had expected that it gave me accurate results, which does not seem to be the case. I am completely new to stats, how are you saying each elements expected value is 1 $\endgroup$ – avz2611 Jul 11 '17 at 4:28
  • $\begingroup$ Calculation of the basic independence chi-squared is here, including how to compute expecteds. No doubt python does give a correct chi-square value, but that may not correspond to the value you need in order to get 1 for $V$, and it's also possible there's some issue with your code even if that's not the problem. If you don't do simple checks of what your code does (and this is an obvious one -- I did it in my head) you may miss all manner of issues. $\endgroup$ – Glen_b -Reinstate Monica Jul 11 '17 at 4:38
  • $\begingroup$ you have any idea why my code is faltering? $\endgroup$ – avz2611 Jul 11 '17 at 4:56
  • $\begingroup$ Step 1 is check all the values I listed are what they should be. If the only one that's different is the chi-squared value, read the documentation on that function -- maybe it makes the continuity correction or something and you need to change an argument to get the unadjusted chi-square. $\endgroup$ – Glen_b -Reinstate Monica Jul 11 '17 at 5:25
  • $\begingroup$ Yes, indeed, it does Yates' correction by default. Again, that's something you should have been telling us, not the other way around. It's up to you to check what the functions you're calling actually say they do (and then to make sure that they seem to be doing what they claim). $\endgroup$ – Glen_b -Reinstate Monica Jul 11 '17 at 5:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.