2
$\begingroup$

I have read that Chi square test is not correct technique for Benford's distribution.

Also, the chi-square test applies when both variables are categorical.

I am little confused if any or both of above statements are true and related? And if so, is it right to use chi sq test for Benford in Audit use cases?

$\endgroup$
1
  • 1
    $\begingroup$ That thread you reference mischaracterizes its source: the criticism of using the chi-squared test for Benford's law is that it lacks power, not that it is in any way "incorrect." It use is perfectly correct, but (slightly) better tests can be devised. $\endgroup$
    – whuber
    Apr 15, 2022 at 18:39

1 Answer 1

3
$\begingroup$

As @whuber explained, the "problem" with use the chi-squared test to assess conformity to Benford's law is that it lacks statistical power. The discrete version of the Kolmogorov-Smirnov goodness-of-fit test (GOF) has better power. There are, however, other alternatives designed especially to test conformity to Benford's law See here for a wider discussion of the matter.

$\endgroup$
3
  • $\begingroup$ Thanks.. can you please share some links demonstrating power of these tests? Also, chi square is used when both variables are categorical, but same is not in case of benford, then how are we applying chi sq to benford ? $\endgroup$ Apr 16, 2022 at 5:24
  • 1
    $\begingroup$ The key problem with power for discrete tests is often conservative nominal values. That's what this paper address and improves upon (see figure 1). I'm not aware of any more extensive power analysis. For the chi-squared test for Benford's law, each digit is treated as a category. Of course, this ignores the fact that these categories are ordered which is why KS or other tests can be more powerful. $\endgroup$
    – num_39
    Apr 16, 2022 at 6:27
  • $\begingroup$ What does it precisely mean? The paper you quote says that the $\chi^2$-test lacks power when the sample size is small but does not really explain why: is it the problem of computing the $p$-values according to the limiting distribution instead of performing the exact test, or something different? In the papers of Perkins-Tygert-Ward that you suggested me in your answer to my question, the exact chi-square seems to be defeated by the root-mean-square for a large number of bins, but Benford's law has only $9$ bins. So, what is really happening? $\endgroup$
    – Plop
    Feb 24, 2023 at 15:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.