Fit power law for distributions with zeroes

Question

I am pretty new to statistics and have some data that I think may follow a power-law distribution. However, it includes zeroes. I understand that mathematically zeroes can't work, but conceptually, would the point of a power law be violated if there are some zeroes in the data? If I were to scale the data by adding 1, would that be fine, or would that throw off everything?

I am sure that this is a very newbie question, but that's what I am, and I would love some help in understanding how power laws work. If the distribution can't be power-law, how can I plot it (in R) to best show its characteristics? Would a log-log plot work?

Thank you in advance.

Answer 1

I guess your "this is a very newbie question" refers to this of your many questions:

"...but conceptually, would the point of a power law be violated if there 
are some zeroes in the data?"**

No. The concept remains valid as the same class of distributions may be applied to data with or without zeros. You may be interested in reading more about Tweedie class of distributions here and then here.

For example, the well-known Taylor’s law says that the variance is proportional to a power of the mean. Taylor’s law is mathematically identical to the variance-to-mean power law that characterizes the Tweedie distributions, that is for any random variable that obeys a Tweedie distribution, the variance relates to the mean by the power law. Since that "any random variable" can be discrete, continuous or a combination of both, the concept of the power law may equally apply to data that are counts (Poisson), reals (Normal), positive reals (Gamma), or positive reals with the added positive mass at zero (compound Poisson–gamma).

Given your "there are some zeros in the data" and your comment "yes, my values are counts", simple Poisson may work. If not, e.g. zeros are too few or too many, you may try Neyman Type A distribution (this R package manual mentions it the context of the Tweedie class of distributions).

I hope some of the above helps.

Answer 2

Let's start with the mathematical form for the power-law distribution:

$p(x)\propto x^{-\alpha}$ for $x\geq x_{\min}>0$ and $\alpha>1$.

As you said, $x=0$ isn't allowed (the reason being that you cannot normalize the function if the range extends down to 0). But note that the distribution is perfectly well-defined for any choice of $x_{\min}>0$, including $x_{\min}=1$.

If your measured quantity sometimes takes the value of $x=0$ then what you can immediately conclude is that your quantity does not follow a power-law distribution over its entire range. But, it could certainly follow a power-law distribution in its upper tail, i.e., for some $x>x_{\min}\geq1$. Often, this is precisely what we mean when we say that such-and-such a quantity follows a power law anyway, which is that we don't really care about the distribution of the small values of $x$, only the shape the tail of the distribution.

Adding 1 to your observed values is not recommended since the 0 values cannot actually be drawn from a power-law distribution, so making them something greater than zero isn't going to help you understand how the rest of your data are distributed. If you really want to include them, you would want to create a piece-wise model, one piece for $x=0$ and one for $x>0$, but that might be a little complicated given your expressed newness to statistics. (Although, it might be really worth it, depending on your ultimate goal with the data.)

For plotting, I would suggest computing the complementary empirical distribution function (which is the fraction of your data set whose value is $\geq X$) and plotting that fuction on log-log axes after removing the point for $X=0$ (since the log-log plot won't like taking the logarithm of 0). This function will start on the $y$-axis at the fraction of your data set that are not 0. If your data are truly generated by a power-law distribution for $x\geq1$, then the plotted function will be a straight line on the log-log axes (WARNING: being straight on log-log axes is a necessary but not a sufficient condition for some data being drawn from a power-law distribution; to really know, you should use real statistical tools like these).

Answer 3

Power-Law distributions (e.g. Zipf's) are supposedly good for modeling certain frequencies of data that follow a descending frequency of occurrence, such as phonemes within a speech or text. The numerical value assigned to certain categories is the order statistic for the frequency in which a certain category occurs, for instance, participles in the English language "a", "the", "an" are usually the first class of words in such a model.

A general graphical tool for measuring distributional assumptions is that of a QQ plot.

If exactly one category has 0 frequencies, then your model can infer what the expected frequency might have been for that variable in resamples of the data based on the estimated power rule. If more than one such category has 0 frequencies, then it's impossible to estimate those values (so cluster them into an "unobserved" category). Otherwise, direct maximum likelihood will be impossible due to ties.

There are advanced methods, however, of estimating statistics derived from order statistics using MCMC methods. In the case of maximum likelihood estimation, it boils down to the EM algorithm. With many ties in a frequency distribution, it is impossible to rank them. One can, however, permute rank assignment over all possible orders in those ties, and take the "expectation" to come up with an estimate of the power law. A similar method is used in Cox models to handle ties, called the Breslow method or the Efron method for ties.