2
$\begingroup$

I'm trying to test if a dataset follows Benford's Law (https://en.wikipedia.org/wiki/Benford%27s_law), which basically says how many values in a data set we'd expect to have a first significant digit (i.e. start with) 1,2,...,9.

Here's some actual data.

1       2       3       4       5       6       7       8       9       FSD
0.301   0.176   0.125   0.097   0.079   0.067   0.058   0.051   0.046   Benford
0.305   0.179   0.126   0.098   0.077   0.064   0.057   0.049   0.046   Observed

As you can see, the observed data is SO close to what Benford expects. I'm trying to argue that Benford is a good model for expectations, but the standard Chi-squared says it is not a good match, since this particular observed data is over 25,000 points. Essentially, the large size of my data set makes the frequency difference look huge. Yet obviously, Benford's Law is a perfect model for this data.

My question: is it statistically correct to do chi-squared with the proportions instead of the frequencies? I know it can be done (I read Can chi square be used to compare proportions?), but I'm more concerned that reviewers of my paper will say that's incorrect.

Improve this question
$\endgroup$
17
  • 4
    $\begingroup$ Your last sentence is puzzling. Chi-square tests of the kind discussed here may easily be recast or presented as testing a hypothesis of some specified set of proportions, but the principle remains at root that they are for counts. If there is debate on this, it's misinformed. Incidentally, testing Benford's Law depends on the data spanning various orders of magnitude. If you have data on adult female height in inches, your first digits are likely to be just 5, 6, 7, and Benford's Law won't hold, but that does not mean that the data are faked. $\endgroup$
    – Nick Cox
    Commented Feb 9, 2015 at 15:57
  • $\begingroup$ What's the P-value? Perhaps you are just getting a signal that Benford's Law is not the only law in town. $\endgroup$
    – Nick Cox
    Commented Feb 9, 2015 at 15:59
  • $\begingroup$ My data is indeed over several orders of magnitude. And as you can see in the sample data I posted, it fits Benford's Law pretty perfectly. For 8 degrees of freedom and with frequencies from my sample size on this data I shared (19,500), the p value is 0.33. $\endgroup$
    – Jay
    Commented Feb 9, 2015 at 16:04
  • 1
    $\begingroup$ The attitude that it's the job of statistically-minded people to provide arguments for the interpretation you prefer is at best dubious and at worst... much worse. The stance should always be: This is my interpretation. Is it statistically sound? I don't want to over-react to your wording, but there is a key principle at stake. $\endgroup$
    – Nick Cox
    Commented Feb 9, 2015 at 16:14
  • 3
    $\begingroup$ Benford provides a near-perfect - but not quite perfect - fit. However, because you have so much data, any decent goodness of fit test can detect that there are tiny deviations. Indeed, you should expect to see consistent deviations in almost any real data set, since the arguments by which Benford's law should hold are not exact for data over finite ranges. Hypothesis tests are not the right tool for the questions of real interest here. $\endgroup$
    – Glen_b
    Commented Feb 9, 2015 at 16:34

1 Answer 1

Reset to default
1
$\begingroup$

Chi-square is not a good test for Benford's law. There is a paper by Morrow (2014), that discusses and compares the performance (convergence) of several test statistics specifically for Benford's law: among them Kolmogorov-Smirnov, Kuiper, Leemis' m, and Euclidean distance dN as formulated by Cho and Gaines (2007). It seems that the better performing tests are Kuiper and dN.

Morrow, J. (2014). Benford’s Law, families of distributions and a test basis [Paper]. Centre for Economic Performance, London School of Economics and Political Science. https://cep.lse.ac.uk/_new/publications/abstract.asp?index=4486

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.