1
$\begingroup$

I'm looking for a normal approximation for a Bernoulli variable (so I can later sum multiple correlated approximated variables)

The trivial approximation is taking the mean and variance of the Bernoulli variable, and use those as parameters for the normal variable.

$$X \sim Bernoulli(p)$$ $$E[X]=p=\mu, Var[X]=p(1-p)=\sigma^2$$ $$\tilde{X} \sim N(\mu,\sigma^2)$$

This approximation has a built-in error (as analyzed here), which might add up when summing the correlated variables.

Is there a better choice for the normal variable parameters that will reduce the error of the approximation?

Improve this question
$\endgroup$
4
  • 1
    $\begingroup$ Why do you want to approximate Bernoulli distribution? $\endgroup$
    – Tim ♦
    Oct 18, 2021 at 7:51
  • $\begingroup$ As I mentioned - I want to sum multiple correlated Bernoulli variables, the summation is hard with Bernoulli distribution, so I'm trying to do that with a normal approximation $\endgroup$
    – user107511
    Oct 18, 2021 at 8:09
  • $\begingroup$ There will be a better choice if you would supply the correlations among your variables. It is also useful to describe what you mean by "the error of the approximation," because how this is quantified and weighted depends on how you intend to use the approximation. $\endgroup$
    – whuber ♦
    Oct 18, 2021 at 15:47
  • $\begingroup$ The correlation is the same for each pair of Bernoulli variables (symmetric problem). It is estimated in simulations, and don't have an exact formula for it. The values are in range $0 < \rho < 0.25$. My final usage of the approximation is taking the sum of the normal variables (which is a normal distributed variable as well), and taking the probability it's smaller than a given constant - means that I'm looking for the probability that the number of successes of all of the Bernoulli variables is smaller than a given constant. $\endgroup$
    – user107511
    Oct 19, 2021 at 7:34

1 Answer 1

Reset to default
1
$\begingroup$

The error should not add up. You get as many times the error positive as negative.

If you are summing the variables that you approximate then in the end you will effectively determine the variance and mean of the final distribution by means of algebraic rules for the mean and variance of adding correlated variables.

$$\mu_{X+Y} = \mu_{X} + \mu_Y$$

$$\text{Var}(X+Y) = \text{Var}(X) + \text{Var}(Y) + 2\text{Cov}(X,Y)$$

and it will be as if you are doing a single normal approximation for the end result. So no errors add up.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Thanks! Couple of questions: 1. Is this argument that you get the same positive and negative errors based on CLT? Isn't it valid only when the variables are independent? 2. Are the algebraic rules valid only for normal distribution? Are there any equivalent rules for Bernoulli sum? $\endgroup$
    – user107511
    Oct 18, 2021 at 11:15
  • $\begingroup$ @user107511 The argument is more indirect. It is based on the idea that you are doing the approximation effectively only once (after figuring out mean and variance of the final sum). So because you are effectively doing the approximation one time, the errors in the sum of multiple approximations should even out instead of sum up. $\endgroup$
    – Sextus Empiricus
    Oct 18, 2021 at 11:18
  • $\begingroup$ Of course the normal approximation can be very bad, but that is due to the correlations and not so much the build up of errors of multiple approximations. Example, let $X\sim Unif(0,1)$ and define $n$ correlated Bernoulli variables with $$Y_i = \begin{cases} 1 & \text{if $i/n\leq X< i/n +0.5)$ } \\ & \text{or $X <i/n-0.5$ }\\ 0 &\text{else}\end{cases}$$ for this case $\sum Y_i = n/2$ $\endgroup$
    – Sextus Empiricus
    Oct 18, 2021 at 11:37
  • $\begingroup$ I can imagine that if you would compute the correlations and apply the formulas for the above example then you get to a variance of zero. You might think that with a variance of the sum that is large then the approximation should be better, but this is not guaranteed. See the example below. $\endgroup$
    – Sextus Empiricus
    Oct 18, 2021 at 11:43
  • $\begingroup$ $$Y_i = \begin{cases} 1 & \text{if $ X\leq i/n $} \\ 0& \text{if $ X> i/n$ }\end{cases}$$ In this case $\sum Y_i$ will approach a triangular distribution. $\endgroup$
    – Sextus Empiricus
    Oct 18, 2021 at 11:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.