# Can a Bernoulli distribution be approximated by a Normal distribution?

$$\sum_{i=1}^n bernoulli(p) = binomial(n,p) \approx \mathcal N(np, np(1-p)) = \sum_{i=1}^n \mathcal N(p, p(1-p))$$

Can I conclude that $$\mathcal N(p, p(1-p))$$ could represent an approximation of $$bernoulli(p)$$?

In particular, given $$n$$ binary RVs $$X_i$$ then a possible naive factorization of $$P(X_1, X_2, \ldots, X_n)$$ is $$P(X_1) P(X_2) \ldots P(X_n)$$.

Since all the RVs are binary, then they can be modeled as Bernoulli RVs.

In case I am not interested in the exact probability of the joint, the can I use the normal distribution to approximate each bernoulli variable?

• Just how poor an approximation can you tolerate? After all, you are proposing to use a continuous distribution to approximate a discrete distribution having two values: except perhaps for some very special calculations, such an approximation would seem to have no advantages. – whuber Dec 5 '18 at 19:52
• When you approximate a binomial with a normal you are using large sample (large n) property. Although there are no hard rules on how large n should be for the sample to be large, I think it is safe to say that a sample of 1 is not large. – Jesper Hybel Dec 5 '18 at 19:52

## 2 Answers

Let's analyze the error.

The figure shows plots of the distribution function of various Bernoulli$$(p)$$ variables in blue and the corresponding Normal distributions in Red. The shaded regions show where the functions differ appreciably.

(Why plot distribution functions instead of density functions? Because a Bernoulli variable has no density function. The densities of good continuous approximations to Bernoulli distributions have huge spikes in neighborhoods of $$0$$ and $$1.$$)

No matter what $$p$$ may be, for some values of $$x$$ the difference between the two distribution functions will be large. After all, the Bernoullli distribution function has two leaps in it: it jumps by $$1-p$$ at $$x=0$$ and again by $$p$$ at $$x=1.$$ The Normal distribution function is going to split the greater of those two leaps into two parts, whence the larger of the two vertical differences--the largest error--must be at least $$1/4.$$ In fact, it's always greater even than that.

Here is a plot of the maximum difference between the two functions, as it depends on $$p:$$

It is never smaller than $$0.341345,$$ attained when $$p=1/2.$$ Because probabilities all lie between $$0$$ and $$1,$$ this is a substantial error. It is difficult to conceive of circumstances where this approximation would be acceptable, except perhaps when $$x\lt 0$$ or $$x\gt 1:$$ but then why use a Normal distribution at all? Just approximate those values as $$0$$ and $$1,$$ respectively, without any error at all.

I don't think you can conclude that N(p,p(1−p)) could represent an approximation of bernoulli(p). First of all, for a bernoulli variable, a random sample could only be 0 or 1, on the other hand, the range of normal variable could be from -inf to inf. Secondly, If we have a random distribution with mean p, and variance p(1-p), once we draw lots of samples from this distribution and add them together, their summation distribution will also follow a normal distribution with mean np and variance np(1-p) due to central limit theorem. For sure we can't say the random distribution represents an approximation of bernoulli(p)...

• The "Secondly" part is unclear, this is what OP is saying, isn't it? Why should this be argument against it? – Tim Dec 5 '18 at 20:56
• Hi Tim, I don't explain it well, I was trying to claim that if the argument is correct, then we could use a random distribution to approximate the bernoulli distribution, which is clearly not correct. let me give you a example, a normal distribution with mean 1 and variance 1, also a poisson distribution with mean 1 and variance 1. the summation of N normal samples is another normal distribution with mean N and variance N, the summation of N poisson samples also follows normal distribution with mean N and variance N. (N is large) but apparently poisson (1) can't approximate normal(1, 1) – Yang Song Dec 6 '18 at 19:41