5
$\begingroup$

$$\sum_{i=1}^n bernoulli(p) = binomial(n,p) \approx \mathcal N(np, np(1-p)) = \sum_{i=1}^n \mathcal N(p, p(1-p))$$

Can I conclude that $\mathcal N(p, p(1-p))$ could represent an approximation of $bernoulli(p)$?

In particular, given $n$ binary RVs $X_i$ then a possible naive factorization of $P(X_1, X_2, \ldots, X_n)$ is $P(X_1) P(X_2) \ldots P(X_n)$.

Since all the RVs are binary, then they can be modeled as Bernoulli RVs.

In case I am not interested in the exact probability of the joint, the can I use the normal distribution to approximate each bernoulli variable?

Improve this question
$\endgroup$
2
  • 3
    $\begingroup$ Just how poor an approximation can you tolerate? After all, you are proposing to use a continuous distribution to approximate a discrete distribution having two values: except perhaps for some very special calculations, such an approximation would seem to have no advantages. $\endgroup$
    – whuber ♦
    Commented Dec 5, 2018 at 19:52
  • 2
    $\begingroup$ When you approximate a binomial with a normal you are using large sample (large n) property. Although there are no hard rules on how large n should be for the sample to be large, I think it is safe to say that a sample of 1 is not large. $\endgroup$
    – Jesper for President
    Commented Dec 5, 2018 at 19:52

2 Answers 2

Reset to default
6
$\begingroup$

Let's analyze the error.

The figure shows plots of the distribution function of various Bernoulli$(p)$ variables in blue and the corresponding Normal distributions in Red. The shaded regions show where the functions differ appreciably.

Figure 1

(Why plot distribution functions instead of density functions? Because a Bernoulli variable has no density function. The densities of good continuous approximations to Bernoulli distributions have huge spikes in neighborhoods of $0$ and $1.$)

No matter what $p$ may be, for some values of $x$ the difference between the two distribution functions will be large. After all, the Bernoullli distribution function has two leaps in it: it jumps by $1-p$ at $x=0$ and again by $p$ at $x=1.$ The Normal distribution function is going to split the greater of those two leaps into two parts, whence the larger of the two vertical differences--the largest error--must be at least $1/4.$ In fact, it's always greater even than that.

Here is a plot of the maximum difference between the two functions, as it depends on $p:$

Figure 2

It is never smaller than $0.341345,$ attained when $p=1/2.$ Because probabilities all lie between $0$ and $1,$ this is a substantial error. It is difficult to conceive of circumstances where this approximation would be acceptable, except perhaps when $x\lt 0$ or $x\gt 1:$ but then why use a Normal distribution at all? Just approximate those values as $0$ and $1,$ respectively, without any error at all.

Improve this answer
$\endgroup$
0
$\begingroup$

I don't think you can conclude that N(p,p(1−p)) could represent an approximation of bernoulli(p). First of all, for a bernoulli variable, a random sample could only be 0 or 1, on the other hand, the range of normal variable could be from -inf to inf. Secondly, If we have a random distribution with mean p, and variance p(1-p), once we draw lots of samples from this distribution and add them together, their summation distribution will also follow a normal distribution with mean np and variance np(1-p) due to central limit theorem. For sure we can't say the random distribution represents an approximation of bernoulli(p)...

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ The "Secondly" part is unclear, this is what OP is saying, isn't it? Why should this be argument against it? $\endgroup$
    – Tim
    Commented Dec 5, 2018 at 20:56
  • $\begingroup$ Hi Tim, I don't explain it well, I was trying to claim that if the argument is correct, then we could use a random distribution to approximate the bernoulli distribution, which is clearly not correct. let me give you a example, a normal distribution with mean 1 and variance 1, also a poisson distribution with mean 1 and variance 1. the summation of N normal samples is another normal distribution with mean N and variance N, the summation of N poisson samples also follows normal distribution with mean N and variance N. (N is large) but apparently poisson (1) can't approximate normal(1, 1) $\endgroup$
    – Yang Song
    Commented Dec 6, 2018 at 19:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.