# perturbing discrete probability distribution

What is the best way to slightly perturb a given discrete probability distribution ?

Adding a zero mean Gaussian noise to the probability distribution and re-normalizing it such that it sums to 1 is one way. But it does not ensure that all the probabilities are positive. Is there an other way to achieve this ?

• Could you please clarify what you mean by "perturb"? Do you want to change the probabilities or the values (or both)? What is the purpose of this perturbation?
– whuber
Nov 14, 2012 at 20:50
• If you provided some motivation we could probably provide better advice. Nov 14, 2012 at 20:58
• I have to echo the other commenters - what is it you want to achieve? Also, do you want to end up with something continuous? discrete? mixed? you don't care? Nov 14, 2012 at 22:09
• The resulting probability distribution should be discrete and only the probabilities should change slightly compared to the original probabilities. The values of the random variable should not change.. Hope it's clearer now.. Nov 15, 2012 at 5:32

I'd look at this problem geometrically, think of the distribution $[p_1, p_2 ... p_N]$ as a point in $N$ dimensional space $\vec{p}$. Now, since all the probabilities are positive and sum to one, actual probability distribution lie on a hyperplane that diagonally spans one "quadrant" in this space (you can probably visualize it in $3D$ with some thought).
Now, you can perturb it by making a randomly selected small displacement in this vector space $\delta \vec{p}$, and then projecting the pertubed point back down to the hyperplane $\sum p_i =1$. The only subtlety would be handling the case where due to the pertubation you end up with $p_i+\delta p_i <0$, which I'd just handle via $p_i' = \lvert p_i + \delta p_i \rvert$, i.e. reflective boundary conditions at $p_i=0$.
The specific distribution for $\delta p_i$ could really be anything, and if you do $M$ small steps via this logic you could get something that could be claimed (in a non-rigorous manner) as being like a Gaussian perturbation.
A natural choice is to use the Dirichlet distribution. $Dir(a p_1, a p_2, \cdots)$ will have mean equal to $(p_1, p_2, \cdots)$ and the variance can be controlled via $a$, and the samples are necessarily distributions.