What's the common way to measure the “stability” or “volatility” of a distribution? Variance seems insufficient

Question

for example, in a discrete distribution A

P(X) = 0.999  when x = 0
P(X) = 0.001  when x = 1000

E(X) = 1
Var(X) = 1000-1=999

compared to B

P(X) = 0.6  when x = 0
P(X) = 0.2  when x = 1
P(X) = 0.2/2  when x = 2
P(X) = 0.2/4  when x = 4
P(X) = 0.2/4  when x = 8

E(X) = 1 =A
Var(X) = 0.2+0.4+0.8+3.2-1=3.6 <<A

However B seems more volatile(instable) than A. What is the common correct stats to measure this sort of thing?

Update:

Entropy might be a good candidate, however it doesn't look into the information itself (it only cares about the chance). I feel the time series graph of a fair coin which could score (0 or 1) is less volatile / more stable than a fair coin which could score (0 or 100).

Answer 1

You have a wide choice of measures of dispersion, and it really depends on how you want to balance the different values that can be taken with different probabilities (as entropy does) against the width of the range of these values (as variance does).

If you wanted something less dramatic than variance because of your concerns about the impact of a rare extreme outlier, you might try the average absolute deviation from the median. For for example A, you would get $$|0-0|\times 0.999 + |1000-0|\times0.001 =1$$ while for example B $$|0-0|\times 0.6 + |1-0|\times 0.2 + |2-0|\times 0.1 + |4-0|\times 0.05 + |8-0|\times 0.05 = 1.$$

It is not quite a coincidence this measure of dispersion is equal to the expectation for these two examples: this will happen for any random-variable where $\Pr(X \lt 0)=0$ and $\Pr(X = 0) \ge \frac12$.

But you should look at other measures of dispersion too.