+1 to @geomatt22 and whuber for their comments and suggestions. Their suggestions work when the information (jumps, first differences, etc.) is not extreme valued. A recent paper by Lin and Tegmark Critical Behavior from Deep Dynamics: A Hidden Dimension in Natural Language (available here ... https://ai2-s2-pdfs.s3.amazonaws.com/5ba0/3a03d844f10d7b4861d3b116818afe2b75f2.pdf), discusses situations that frequently occur where the information is power-lawed, extreme valued and exhibits critical complexity. In particular, they take "traditional" sequential analysis based on Markov processes to task for being "shallow" and unable to capture deep, long-term correlations. Here is their abstract:
We show that in many data sequences — from texts in different
languages to melodies and genomes — the mutual information between two
symbols decays roughly like a power law with the number of symbols in
between the two. In contrast, we prove that Markov/hidden Markov
processes generically exhibit exponential decay in their mutual
information, which explains why natural languages are poorly
approximated by Markov processes. We present a broad class of models
that naturally reproduce this critical behavior. They all involve deep
dynamics of a recursive nature, as can be approximately implemented by
tree-like or recurrent deep neural networks. This model class captures
the essence of probabilistic context-free grammars as well as
recursive self-reproduction in physical phenomena such as turbulence
and cosmological inflation. We derive an analytic formula for the
asymptotic power law and elucidate our results in a statistical
physics context: 1-dimensional “shallow” models (such as Markov models
or regular grammars) will fail to model natural language, because they
cannot exhibit criticality, whereas “deep” models with one or more
“hidden” dimensions representing levels of abstraction or scale can
potentially succeed.
This isn't intended to suggest that GeoMatt22 and Whuber's suggestions are wrong, it's merely intended to suggest an alternative formulation of the problem.
In addition to Tegmark's work, another recent paper by J.P. Bouchaud Crises and Collective Socio-Economic Phenomena (available here ... https://www.cfm.fr/assets/ResearchPapers/Crises+and+collective+socio-economic+phenomena.pdf) specifically discusses modeling behaviors such as sudden ruptures, crises and avalanches, which are close analogues to burstiness. Here is their abstract:
Financial and economic history is strewn with bubbles and crashes,
booms and busts, crises and upheavals of all sorts. Understanding the
origin of these events is arguably one of the most important problems
in economic theory. In this paper, we review recent efforts to include
heterogeneities and interactions in models of decision. We argue that
the so-called Random Field Ising model (rfim) provides a unifying
framework to account for many collective socio-economic phenomena that
lead to sudden ruptures and crises. We discuss different models that
can capture potentially destabilising self-referential feedback loops,
induced either by herding, i.e. reference to peers, or trending, i.e.
reference to the past, and that account for some of the phenomenology
missing in the standard models. We discuss some empirically testable
predictions of these models, for example robust signatures of
rfim-like herding effects, or the logarithmic decay of spatial
correlations of voting patterns. One of the most striking result,
inspired by statistical physics methods, is that Adam Smith’s
invisible hand can fail badly at solving simple coordination problems.
We also insist on the issue of time-scales, that can be extremely long
in some cases, and prevent socially optimal equilibria from being
reached. As a theoretical challenge, the study of so-called
“detailed-balance” violating decision rules is needed to decide
whether conclusions based on current models (that all assume
detailed-balance) are indeed robust and generic.
Together, these two papers represent significant advances in the analysis of extreme valued behaviors.
To be specific wrt your question about developing a "single" value to compare the two distributions, there are certainly many ways to do this. One way that incorporates the possibility of the information being extreme valued is to estimate the tail index of the distribution -- whether raw data, first differences, whatever. One easily generated approach to tail estimation is explained in Gabaix's paper on OLS modeling of the log-ranks (available here ... http://www.eco.uc3m.es/temp/jbes.2009.06157.pdf) or by leveraging the more rigorous and computationally intensive methods developed by Pickands or Hill. Once an index is available, then a distribution can be assigned based on the Examples section of this Wiki discussion of the Tweedie family of distributions... https://en.wikipedia.org/wiki/Tweedie_distribution
