I am pretty new to statistics and have some data that I think may follow a power-law distribution. However, it includes zeroes. I understand that mathematically zeroes can't work, but conceptually, would the point of a power law be violated if there are some zeroes in the data? If I were to scale the data by adding 1, would that be fine, or would that throw off everything?

I am sure that this is a very newbie question, but that's what I am, and I would love some help in understanding how power laws work. If the distribution can't be power-law, how can I plot it (in R) to best show its characteristics? Would a log-log plot work?

Thank you in advance.

  • 1
    $\begingroup$ You could consider using a zero-inflated model. $\endgroup$ – david25272 Jan 22 '14 at 22:40
  • $\begingroup$ I'm sorry, I'm really very new. Do you mean I can use a zero-inflated model to fit a power-law, or that I can use one to plot my data? $\endgroup$ – bsg Jan 22 '14 at 22:45
  • $\begingroup$ A zero inflated model is a two stage model, the first stage models the number of zeros, and the second stage is a Poisson distribution, or some other distribution. It seems that you have a "too many zeros" problem, which is why I suggested it. $\endgroup$ – david25272 Jan 22 '14 at 22:55
  • $\begingroup$ (1) How do the zeros arise? Are your values counts? $\quad\quad\\$ (2) A power law is presumably a model for the expectation of some response ($y$) in terms of some predictor, ($x$). I presume your zeros are $y$ values, rather than $x$ values, in which case that may present no problem for the model (you might have a zero observation with a nonzero expectation), though it may give you trouble if you want to do a log-log plot. $\quad\quad\\$ (3) On the other hand if it's the $x$ that's zero when $y$ isn't, your power-law needs to be modified. $\endgroup$ – Glen_b -Reinstate Monica Jan 22 '14 at 23:08
  • 1
    $\begingroup$ Yes, my values are counts. $\endgroup$ – bsg Jan 22 '14 at 23:14

I guess your "this is a very newbie question" refers to this of your many questions:

"...but conceptually, would the point of a power law be violated if there 
are some zeroes in the data?"**

No. The concept remains valid as the same class of distributions may be applied to data with or without zeros. You may be interested in reading more about Tweedie class of distributions here and then here.

For example, the well-known Taylor’s law says that the variance is proportional to a power of the mean. Taylor’s law is mathematically identical to the variance-to-mean power law that characterizes the Tweedie distributions, that is for any random variable that obeys a Tweedie distribution, the variance relates to the mean by the power law. Since that "any random variable" can be discrete, continuous or a combination of both, the concept of the power law may equally apply to data that are counts (Poisson), reals (Normal), positive reals (Gamma), or positive reals with the added positive mass at zero (compound Poisson–gamma).

Given your "there are some zeros in the data" and your comment "yes, my values are counts", simple Poisson may work. If not, e.g. zeros are too few or too many, you may try Neyman Type A distribution (this R package manual mentions it the context of the Tweedie class of distributions).

I hope some of the above helps.

  • $\begingroup$ Thank you! I think I will have to read up on some of the concepts you mentioned. $\endgroup$ – bsg Jan 24 '14 at 1:42

Let's start with the mathematical form for the power-law distribution:

$p(x)\propto x^{-\alpha}$ for $x\geq x_{\min}>0$ and $\alpha>1$.

As you said, $x=0$ isn't allowed (the reason being that you cannot normalize the function if the range extends down to 0). But note that the distribution is perfectly well-defined for any choice of $x_{\min}>0$, including $x_{\min}=1$.

If your measured quantity sometimes takes the value of $x=0$ then what you can immediately conclude is that your quantity does not follow a power-law distribution over its entire range. But, it could certainly follow a power-law distribution in its upper tail, i.e., for some $x>x_{\min}\geq1$. Often, this is precisely what we mean when we say that such-and-such a quantity follows a power law anyway, which is that we don't really care about the distribution of the small values of $x$, only the shape the tail of the distribution.

Adding 1 to your observed values is not recommended since the 0 values cannot actually be drawn from a power-law distribution, so making them something greater than zero isn't going to help you understand how the rest of your data are distributed. If you really want to include them, you would want to create a piece-wise model, one piece for $x=0$ and one for $x>0$, but that might be a little complicated given your expressed newness to statistics. (Although, it might be really worth it, depending on your ultimate goal with the data.)

For plotting, I would suggest computing the complementary empirical distribution function (which is the fraction of your data set whose value is $\geq X$) and plotting that fuction on log-log axes after removing the point for $X=0$ (since the log-log plot won't like taking the logarithm of 0). This function will start on the $y$-axis at the fraction of your data set that are not 0. If your data are truly generated by a power-law distribution for $x\geq1$, then the plotted function will be a straight line on the log-log axes (WARNING: being straight on log-log axes is a necessary but not a sufficient condition for some data being drawn from a power-law distribution; to really know, you should use real statistical tools like these).

  • $\begingroup$ How do you plot the "complementary empirical distribution function"? $\endgroup$ – FaCoffee May 12 '16 at 10:43

Power-Law distributions (e.g. Zipf's) are supposedly good for modeling certain frequencies of data that follow a descending frequency of occurrence, such as phonemes within a speech or text. The numerical value assigned to certain categories is the order statistic for the frequency in which a certain category occurs, for instance, participles in the English language "a", "the", "an" are usually the first class of words in such a model.

A general graphical tool for measuring distributional assumptions is that of a QQ plot.

If exactly one category has 0 frequencies, then your model can infer what the expected frequency might have been for that variable in resamples of the data based on the estimated power rule. If more than one such category has 0 frequencies, then it's impossible to estimate those values (so cluster them into an "unobserved" category). Otherwise, direct maximum likelihood will be impossible due to ties.

There are advanced methods, however, of estimating statistics derived from order statistics using MCMC methods. In the case of maximum likelihood estimation, it boils down to the EM algorithm. With many ties in a frequency distribution, it is impossible to rank them. One can, however, permute rank assignment over all possible orders in those ties, and take the "expectation" to come up with an estimate of the power law. A similar method is used in Cox models to handle ties, called the Breslow method or the Efron method for ties.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.