I'm trying to test if a dataset follows Benford's Law (https://en.wikipedia.org/wiki/Benford%27s_law), which basically says how many values in a data set we'd expect to have a first significant digit (i.e. start with) 1,2,...,9.

Here's some actual data.

1       2       3       4       5       6       7       8       9       FSD
0.301   0.176   0.125   0.097   0.079   0.067   0.058   0.051   0.046   Benford
0.305   0.179   0.126   0.098   0.077   0.064   0.057   0.049   0.046   Observed

As you can see, the observed data is SO close to what Benford expects. I'm trying to argue that Benford is a good model for expectations, but the standard Chi-squared says it is not a good match, since this particular observed data is over 25,000 points. Essentially, the large size of my data set makes the frequency difference look huge. Yet obviously, Benford's Law is a perfect model for this data.

My question: is it statistically correct to do chi-squared with the proportions instead of the frequencies? I know it can be done (I read Can chi square be used to compare proportions?), but I'm more concerned that reviewers of my paper will say that's incorrect.

  • 2
    $\begingroup$ Your last sentence is puzzling. Chi-square tests of the kind discussed here may easily be recast or presented as testing a hypothesis of some specified set of proportions, but the principle remains at root that they are for counts. If there is debate on this, it's misinformed. Incidentally, testing Benford's Law depends on the data spanning various orders of magnitude. If you have data on adult female height in inches, your first digits are likely to be just 5, 6, 7, and Benford's Law won't hold, but that does not mean that the data are faked. $\endgroup$ – Nick Cox Feb 9 '15 at 15:57
  • $\begingroup$ What's the P-value? Perhaps you are just getting a signal that Benford's Law is not the only law in town. $\endgroup$ – Nick Cox Feb 9 '15 at 15:59
  • $\begingroup$ My data is indeed over several orders of magnitude. And as you can see in the sample data I posted, it fits Benford's Law pretty perfectly. For 8 degrees of freedom and with frequencies from my sample size on this data I shared (19,500), the p value is 0.33. $\endgroup$ – Jay Feb 9 '15 at 16:04
  • 1
    $\begingroup$ The attitude that it's the job of statistically-minded people to provide arguments for the interpretation you prefer is at best dubious and at worst... much worse. The stance should always be: This is my interpretation. Is it statistically sound? I don't want to over-react to your wording, but there is a key principle at stake. $\endgroup$ – Nick Cox Feb 9 '15 at 16:14
  • 2
    $\begingroup$ Benford provides a near-perfect - but not quite perfect - fit. However, because you have so much data, any decent goodness of fit test can detect that there are tiny deviations. Indeed, you should expect to see consistent deviations in almost any real data set, since the arguments by which Benford's law should hold are not exact for data over finite ranges. Hypothesis tests are not the right tool for the questions of real interest here. $\endgroup$ – Glen_b Feb 9 '15 at 16:34

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.