7
$\begingroup$

Suppose I have data of a certain quantity $X$, given by $x_1,...,x_n$. Now, I take the first digit $d_i$ of each quantity $x_i$, and I want to study the relationship between the empirical distribution of first digits $\hat{p}=(\hat{p}_1,...,\hat{p}_n)$ where $\hat{p_i}$ is the normalized frequency of ocurrence of $i$ as the first digit, and benford's law $$ p_i = \log_{10} (1 + 1/i) $$ Now, I've read this paper on the subject of comparing empirical frequencies of first digits vs benford's law. However, they don't mention whether the methods they mention can be used to be able to reject benford's law with a certain confidence in real time, where data arrive with a certain frequency (like, say, 50 data per second).

I think these methods can be applied to real time comparison with benford law in the following way: given a (small) interval of time (say, 3 seconds), we compute the empirical frecuencies of first digits $\hat{p}=(\hat{p}_1,...,\hat{p}_n)$ and then we compute the simultaneous confidence intervals and $p$-values of the statistics shown in the reference I mentioned previously (we have to make sure to have a sample size of at least 60 data, so that the distribution of the statistics should be relatively close to the asymptotic distributions, so the computed $p$-values should be reliable).

My question is, is this a valid procedure? Does it make sense? If not, is there some sound method for comparing empirical first digit distribution with benford law in real time?

One potential problem I see is that the underlying distribution of first digits may change in a given time window (perhaps even more than once). Which is why I think it is a good idea to use relatively small time windows, so as to have a decent sample size, while reducing the odds that the underlying distribution of first digits may change.

$\endgroup$
  • 3
    $\begingroup$ +1 It's a good question--but the proposed procedure is invalid. Confidence intervals cannot be used for sequential tests; they will give too many false alarms. $\endgroup$ – whuber Sep 26 '16 at 16:42
  • $\begingroup$ @whuber thanks!. So I pressume goodness of fit tests would be invalid in this case as well, for the same reason? $\endgroup$ – Nate River Sep 26 '16 at 18:28
  • $\begingroup$ Yes, I think that's right. $\endgroup$ – whuber Sep 26 '16 at 19:07
  • 1
    $\begingroup$ My initial intuition is that some kind of Bayesian approach would be most natural? Eg. Bayesian techniques were used by Jean Baptiste Eugène Estienne to test batches of ammunition while wasting fewer cartridges. Fewer wasted cartridges would loosely correspond to less time in your problem. $\endgroup$ – Matthew Gunn Sep 26 '16 at 22:53
  • $\begingroup$ You may find some value in exploring sequential analysis, which relates to serial testing problems. $\endgroup$ – Glen_b -Reinstate Monica Dec 8 '19 at 8:08
0
$\begingroup$

Sambridge et al. (2010) outline a method for assessing the conformance of time series data to Benford's law. Although your use case is a bit different, it may work for you too.

Their method works as you basically describe: group your data into observation windows and test each window for conformance. This method has been used (and published) by the same authors in other articles, so it's at least sound enough to pass peer review a few times.

Although they have their own goodness of fit measure, I don't see any reason why you couldn't use any measure that would typically work for benford's analysis. You will want to be sure that your measure has good properties for the window or sample size you selected.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.