perturbing discrete probability distribution What is the best way to slightly perturb a given discrete probability distribution ? 
Adding a zero mean Gaussian noise to the probability distribution and re-normalizing it such that it sums to 1 is one way. But it does not ensure that all the probabilities are positive. Is there an other way to achieve this ?
 A: A natural choice is to use the Dirichlet distribution. $Dir(a p_1, a p_2, \cdots)$ will have mean equal to $(p_1, p_2, \cdots)$ and the variance can be controlled via $a$, and the samples are necessarily distributions.
A: I'd look at this problem geometrically, think of the
distribution  $[p_1, p_2 ... p_N]$ as a point in $N$ dimensional
space $\vec{p}$.  Now, since all the probabilities are positive and sum to
one, actual probability distribution lie on a hyperplane that 
diagonally spans one "quadrant" in this space (you can probably visualize it in $3D$ with some thought).
Now, you can perturb it by making a randomly selected small displacement in this 
vector space $\delta \vec{p}$, and then projecting the pertubed point back down to the hyperplane $\sum p_i =1$.  The only subtlety would be handling the case where due to the pertubation you end up with $p_i+\delta p_i <0$, which I'd just
handle via $p_i' = \lvert p_i + \delta p_i \rvert$, i.e. reflective boundary conditions at $p_i=0$.
The specific distribution for $\delta p_i$ could really be anything, and if you do $M$ small steps via this logic you could get something that could be claimed 
(in a non-rigorous manner) as being like a Gaussian perturbation. 
