Intuitive descriptive statistics for power law distributions

Question

For normal distributions we have a set of intuitive descriptive statistics that help us understand how a variable is distributed (mean, mode, median, standard deviation, skewness, & kurtosis). What are similarly intuitive descriptive statistics for characterizing a power-law distribution? Which of the above mentioned descriptive statistics are also applicable to power-law distributions?

Answer 1

The Pareto distribution is a common example of a power law, it is characterized by its scale $x_m>0$ and shape $\alpha > 0$.

However, there is something special about the mean and variance of the normal distribution. The normal distribution belongs to the exponential family. This means that the parameters which maximize the likelihood of a sample can be inferred from the mean and variance, which are - in this case - a sufficient statistic. This means that if you want the maximum likelihood fit of a normal law is completely determined by the mean and variance of the sample, nothing else!

The sufficient statistic for the Pareto distribution is given by

• the expectation of the log $\frac{1}{N}\sum_{i=1}^n \log x_i$
• the smallest element, $\min\limits_i x_i $

Intuitively, this tells you that the relevant information in a Pareto distribution isn't the mean, but the mean of the log. This is why power laws typically describes "orders of magnitude".

If you take a sample distributed according to a Pareto distribution, and compute, on average, by how many orders of magnitude large elements are bigger than the smallest element. For instance, if your sample is a list of city population sizes, then, on average, are cities two orders of magnitudes bigger than the smallest city, three orders of magnitude?

$\alpha$, the shape parameter can be estimated as the inverse of that average.