Skip to main content
added 17 characters in body
Source Link
Patrick Caldon
  • 1.6k
  • 10
  • 11

In short, no.

First, we expect around 30% of the numbers to be 1s.

Second, my guess is that you really want to know if the numbers were "generated by a natural process" or generated artifically, because that's what people use Benford's law for. Benford's law won't tell you much/anything on amany kinds of time series data.

We have strong reason to believe the revenue of particular companies is autocorrelated year by year. If you look at first digits of an autocorrelated sequence, it oftentimes won't follow Benford's law despite being "generated by nature". I guess you could difference the sequence, but I don't know what a "(non?)-Benford's-law compliant first differenced sequence" means, I suspect not a lot.

How you might want to use Benford's law:

You should look at a chi-squared statistic for the expected (benford) distribution of digits and the real distribution, as a measure of the degree to which the discoveredreal distribution satisfies Benford's law.

You can do statistical tests on this, but I don't think these add a lot of value - the real-world distribution might not correspond to Benford's law for a lot of reasons. Where the technique works ok/well is outlier detection - is when you have a population of distributions, say generated by different people. You can then get assurance from the population as a whole that the distribution "ought" to follow Benford's law to a reasonable degree, and look at outliers which satisfy Benford's law to a much lesser degree.

It works well on this intuitive/exploratory level. I've not seen anyone build more formal models which really worked effectively, but that might just be my ignorance.

In short, no.

First, we expect around 30% of the numbers to be 1s.

Second, my guess is that you really want to know if the numbers were "generated by a natural process" or generated artifically, because that's what people use Benford's law for. Benford's law won't tell you much/anything on a time series.

We have strong reason to believe the revenue of particular companies is autocorrelated year by year. If you look at first digits of an autocorrelated sequence, it oftentimes won't follow Benford's law despite being "generated by nature". I guess you could difference the sequence, but I don't know what a "(non?)-Benford's-law compliant first differenced sequence" means, I suspect not a lot.

How you might want to use Benford's law:

You should look at a chi-squared statistic for the expected (benford) distribution of digits and the real distribution, as a measure of the degree to which the discovered distribution satisfies Benford's law.

You can do statistical tests on this, but I don't think these add a lot of value - the real-world distribution might not correspond to Benford's law for a lot of reasons. Where the technique works ok/well is outlier detection - is when you have a population of distributions, say generated by different people. You can then get assurance from the population as a whole that the distribution "ought" to follow Benford's law to a reasonable degree, and look at outliers which satisfy Benford's law to a much lesser degree.

It works well on this intuitive/exploratory level. I've not seen anyone build more formal models which really worked effectively, but that might just be my ignorance.

In short, no.

First, we expect around 30% of the numbers to be 1s.

Second, my guess is that you really want to know if the numbers were "generated by a natural process" or generated artifically, because that's what people use Benford's law for. Benford's law won't tell you much/anything on many kinds of time series data.

We have strong reason to believe the revenue of particular companies is autocorrelated year by year. If you look at first digits of an autocorrelated sequence, it oftentimes won't follow Benford's law despite being "generated by nature". I guess you could difference the sequence, but I don't know what a "(non?)-Benford's-law compliant first differenced sequence" means, I suspect not a lot.

How you might want to use Benford's law:

You should look at a chi-squared statistic for the expected (benford) distribution of digits and the real distribution, as a measure of the degree to which the real distribution satisfies Benford's law.

You can do statistical tests on this, but I don't think these add a lot of value - the real-world distribution might not correspond to Benford's law for a lot of reasons. Where the technique works ok/well is outlier detection - is when you have a population of distributions, say generated by different people. You can then get assurance from the population as a whole that the distribution "ought" to follow Benford's law to a reasonable degree, and look at outliers which satisfy Benford's law to a much lesser degree.

It works well on this intuitive/exploratory level. I've not seen anyone build more formal models which really worked effectively, but that might just be my ignorance.

Source Link
Patrick Caldon
  • 1.6k
  • 10
  • 11

In short, no.

First, we expect around 30% of the numbers to be 1s.

Second, my guess is that you really want to know if the numbers were "generated by a natural process" or generated artifically, because that's what people use Benford's law for. Benford's law won't tell you much/anything on a time series.

We have strong reason to believe the revenue of particular companies is autocorrelated year by year. If you look at first digits of an autocorrelated sequence, it oftentimes won't follow Benford's law despite being "generated by nature". I guess you could difference the sequence, but I don't know what a "(non?)-Benford's-law compliant first differenced sequence" means, I suspect not a lot.

How you might want to use Benford's law:

You should look at a chi-squared statistic for the expected (benford) distribution of digits and the real distribution, as a measure of the degree to which the discovered distribution satisfies Benford's law.

You can do statistical tests on this, but I don't think these add a lot of value - the real-world distribution might not correspond to Benford's law for a lot of reasons. Where the technique works ok/well is outlier detection - is when you have a population of distributions, say generated by different people. You can then get assurance from the population as a whole that the distribution "ought" to follow Benford's law to a reasonable degree, and look at outliers which satisfy Benford's law to a much lesser degree.

It works well on this intuitive/exploratory level. I've not seen anyone build more formal models which really worked effectively, but that might just be my ignorance.