# perturbing discrete probability distribution

What is the best way to slightly perturb a given discrete probability distribution ?

Adding a zero mean Gaussian noise to the probability distribution and re-normalizing it such that it sums to 1 is one way. But it does not ensure that all the probabilities are positive. Is there an other way to achieve this ?

-
Could you please clarify what you mean by "perturb"? Do you want to change the probabilities or the values (or both)? What is the purpose of this perturbation? –  whuber Nov 14 '12 at 20:50
If you provided some motivation we could probably provide better advice. –  Dason Nov 14 '12 at 20:58
I have to echo the other commenters - what is it you want to achieve? Also, do you want to end up with something continuous? discrete? mixed? you don't care? –  Glen_b Nov 14 '12 at 22:09
The resulting probability distribution should be discrete and only the probabilities should change slightly compared to the original probabilities. The values of the random variable should not change.. Hope it's clearer now.. –  locke14 Nov 15 '12 at 5:32

I'd look at this problem geometrically, think of the distribution $[p_1, p_2 ... p_N]$ as a point in $N$ dimensional space $\vec{p}$. Now, since all the probabilities are positive and sum to one, actual probability distribution lie on a hyperplane that diagonally spans one "quadrant" in this space (you can probably visualize it in $3D$ with some thought).

Now, you can perturb it by making a randomly selected small displacement in this vector space $\delta \vec{p}$, and then projecting the pertubed point back down to the hyperplane $\sum p_i =1$. The only subtlety would be handling the case where due to the pertubation you end up with $p_i+\delta p_i <0$, which I'd just handle via $p_i' = \lvert p_i + \delta p_i \rvert$, i.e. reflective boundary conditions at $p_i=0$.

The specific distribution for $\delta p_i$ could really be anything, and if you do $M$ small steps via this logic you could get something that could be claimed (in a non-rigorous manner) as being like a Gaussian perturbation.

-
Thanks.. It was the boundary condition that I didn't know how to handle when it is negative.. –  locke14 Nov 15 '12 at 5:27

A natural choice is to use the Dirichlet distribution. $Dir(a p_1, a p_2, \cdots)$ will have mean equal to $(p_1, p_2, \cdots)$ and the variance can be controlled via $a$, and the samples are necessarily distributions.

-
Thanks. This is an alternative way to an additive perturbation. In this solution there are no worries about negative values.. –  locke14 Nov 15 '12 at 5:29