Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

What is the best way to slightly perturb a given discrete probability distribution ?

Adding a zero mean Gaussian noise to the probability distribution and re-normalizing it such that it sums to 1 is one way. But it does not ensure that all the probabilities are positive. Is there an other way to achieve this ?

share|improve this question
Could you please clarify what you mean by "perturb"? Do you want to change the probabilities or the values (or both)? What is the purpose of this perturbation? – whuber Nov 14 '12 at 20:50
If you provided some motivation we could probably provide better advice. – Dason Nov 14 '12 at 20:58
I have to echo the other commenters - what is it you want to achieve? Also, do you want to end up with something continuous? discrete? mixed? you don't care? – Glen_b Nov 14 '12 at 22:09
The resulting probability distribution should be discrete and only the probabilities should change slightly compared to the original probabilities. The values of the random variable should not change.. Hope it's clearer now.. – locke14 Nov 15 '12 at 5:32
up vote 2 down vote accepted

I'd look at this problem geometrically, think of the distribution $[p_1, p_2 ... p_N]$ as a point in $N$ dimensional space $\vec{p}$. Now, since all the probabilities are positive and sum to one, actual probability distribution lie on a hyperplane that diagonally spans one "quadrant" in this space (you can probably visualize it in $3D$ with some thought).

Now, you can perturb it by making a randomly selected small displacement in this vector space $\delta \vec{p}$, and then projecting the pertubed point back down to the hyperplane $\sum p_i =1$. The only subtlety would be handling the case where due to the pertubation you end up with $p_i+\delta p_i <0$, which I'd just handle via $p_i' = \lvert p_i + \delta p_i \rvert$, i.e. reflective boundary conditions at $p_i=0$.

The specific distribution for $\delta p_i$ could really be anything, and if you do $M$ small steps via this logic you could get something that could be claimed (in a non-rigorous manner) as being like a Gaussian perturbation.

share|improve this answer
Thanks.. It was the boundary condition that I didn't know how to handle when it is negative.. – locke14 Nov 15 '12 at 5:27

A natural choice is to use the Dirichlet distribution. $Dir(a p_1, a p_2, \cdots)$ will have mean equal to $(p_1, p_2, \cdots)$ and the variance can be controlled via $a$, and the samples are necessarily distributions.

share|improve this answer
Thanks. This is an alternative way to an additive perturbation. In this solution there are no worries about negative values.. – locke14 Nov 15 '12 at 5:29

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.