Can I use likelihood-ratio test to compare two samples drawn from power-law distributions?

Question

I have to compare two large samples ($N = 10^{6}$) of discrete data drawn from power-law distributions to assess whether they are significantly different. I can't do that by means of a two-sample Kolmogorov-Smirnov test because my data are discrete. I was wondering if I could do something different. In particular, I would like to apply the likelihood-ratio test in the following way.

Suppose that I have two large samples drawn from two power-law distributions, $s_{1} \sim p(\alpha)$ and $s_{2} \sim p(\alpha)$, and I want to assess if the difference between the estimated tail exponents, $\hat{\alpha}_{1}$ and $\hat{\alpha}_{2}$, is statistically significant --- i.e., if there is a significant difference between the two samples.

My idea was to build a likelihood-ratio test

$\Lambda = -2\times l(H_{0}|s_{1},s_{2}) + 2\times \left[l(H_{1}|s_{1}) + l(H_{1}|s_{2})\right],$

where $l(H_{0}|s_{1},s_{2})$, i.e. the log-likelihood of the null model, is the log-likelihood of the pooled samples $s_{1}, s_{2}$, whereas $l(H_{1}|s_{1}) + l(H_{1}|s_{2})$, i.e. the log-likelihood of the alternative model, is the sum of the log-likelihoods of the samples $s_{1}$ and $s_{2}$.

Then, I would compare the test statistics $\Lambda$ with the $\chi^{2}$ distribution with degrees of freedom $\mathtt{df} = 2 - 1 = 1$, because in the alternative model I need to estimate two parameters (one for sample), whereas in the null model, since the samples are pooled, I need to estimate only one parameter.

Does it make sense? Or should anyone revoke my M.Sc. in Statistics? :)

Otherwise, can anyone suggest more methods to compare two large samples ($N = 10^6$) of discrete data drawn from power-law distributions?

Thanks!

Answer 1

What you have certainly works. Another option that would only require you to only run the non-pooled model (where you estimate both $\hat\alpha_1$ and $\hat\alpha_2$) is the Wald test with the linear hpothesis

$$ H_o: \alpha_1 - \alpha_2 =0 $$ $$ H_1: \alpha_1 - \alpha_2 \neq 0 $$

If your sample size is large, then this method may be more efficient from a computational standpoint (since you only have to run one model instead of two). Other than that both the likelihood ratio and Wald tests are equivalent asymptotically.

Answer 2

I can't do that by means of Kolmogorov-Smirnov test because my data are discrete,

Well, actually you could use a Komogorov-Smirnov test on discrete data as long as either:

(i) you don't use the distribution of the test statistic that assumes the data are continuous. You could, for example, run a permutation or randomization test on the data you have, and you could use the K-S statistic for that if you wanted. This would deal with the impact of discreteness on the distribution of the test statistic.

(ii) you are prepared to deal with the consequences of ignoring the discreteness (lower-than-nominal significance level and corresponding reduction in power) and using the tables anyway. With a sample size of a million, that may not actually be such a problem; you can always use simulation to get a sense of where your actual significance lies. It largely depends on "how discrete" the discrete distribution is.

---

That said, a likelihood ratio test makes perfect sense, too (but how do you know for sure you have a power-law?).

You would indeed proceed exactly as you've said. In small samples, you might try to work out the exact small sample distribution of some simple transformation of the LRT, but with a huge sample there's no reason to bother with all that.

(If your distribution were to have more parameters than the one you mention, under the formulation you give, any additional parameters are assumed constant across samples.)

I suggest taking a look at the paper by Clauset, Shalizi and Newman (2009) [1], which to my recollection covers both continuous and discrete power laws and discusses both Kolmogorov-Smirnov and likelihood ratio tests.

[1] Aaron Clauset, Cosma Rohilla Shalizi, M. E. J. Newman (2009),
"Power-law distributions in empirical data,"
SIAM Review 51, 661-703
(also arXiv:0706.1062v2)