0
$\begingroup$

We know a series of probability distribution approximations that are considered good as long as some condition holds. A few examples are:

  • Binomial can be approximated by Normal if $np(1-p) > 10$ and
  • Beta can be approximated by Normal if $\alpha,\beta$ are large and similar
  • Binomial can be approximated by Poisson if $n$ is large and $p$ is small

Now my question is, how can we quantify if an approximation is "good"? I guess those thresholds are somehow arbitrary, but also that I can measure some quantity that can indicate how robust is the given approximation. What would be the correct metric to check when looking for a good approximation?

For example, let's say that I want to approximate a Beta with a Normal. I know that $\mu = \frac{\alpha}{\alpha+\beta}$ and $\sigma = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)}$.

Let's say $\alpha = 100, \beta = 10$, then I can write:

alpha = 100
beta = 10

mu = alpha/(alpha+beta)
sd = sqrt((alpha*beta)/((alpha+beta)^2 * (alpha+beta+1)))


bb = dbeta(seq(0, 1, 0.001), alpha, beta)
nn = dnorm(seq(0, 1, 0.001), mu, sd)

plot(bb, type = "l", col = "red")
lines(nn, col = "blue")

PDFs. Red: real, Blue: approximation

This seems to be fairly good. Switching to $\alpha = 10, \beta = 1$ leads to a very poor approximation instead.

PDFs. Red: real, Blue: approximation

So I wonder whether there are established procedures to quantify this. I imagine that an approximation can be good in some contexts and not in others, so I would like to understand what is the go-to general quantitative framework to evaluate this in each case.

What would you check and how would you decide if an approximation of a PDF is good or not?

Thank you

EDIT

As there is no universal criterion to define a good approximation, I define my particular focus in the deconvolution problem.

Taking the Beta/Normal case as an example, I am concerned about the goodness of approximation that I would obtain using two normals as Beta approximations and convolving them, compared to directly computing the convolution of Betas. I think this could be addressed with some simulations.

$\endgroup$
4
  • $\begingroup$ A small Kolmogorov–Smirnov statistic is one possible measure: if the two cumulative distribution functions are $F(x)$ and $G(x)$ then you want $\max\limits_x |F(x)-G(x)|$ to be small. It usual use is to compare sample data to a specified distribution, but you can also use it for two distributions. Comparing densities does not work: Poisson or binomial distributions do not have a density $\endgroup$
    – Henry
    Jul 10, 2022 at 17:42
  • 4
    $\begingroup$ The answer depends on your needs: what are you going to use the approximation for? For example, approximations used to compute p-values need to give accurate tail values between the $0.0001$ and $0.1$ and $0.9$ and $0.9999$ quantiles, usually--but accuracy in the middle would be irrelevant. Approximations for other purposes require other measures of accuracy. There is no "go-to general quantitative framework." $\endgroup$
    – whuber
    Jul 10, 2022 at 19:54
  • $\begingroup$ Thank you for the comment @whuber. One application is for sure the p-value. The one that made me ask the question is the convolution problem: for example, the convolution of two Beta distributions is not a Beta itself and it's harder to deal with, but if the two can be approximated to normal then the convolution becomes an easy task. So if there is not a general approach, the strategy could be to compute the real convolution numerically and the approx convolution with different parameters, and see how well they match. I will update the post with this information. $\endgroup$
    – gianMa
    Jul 10, 2022 at 20:10
  • $\begingroup$ But again--we need you to tell us in what way the approximation needs to work well and how well it needs to work. $\endgroup$
    – whuber
    Jul 10, 2022 at 21:21

1 Answer 1

2
$\begingroup$

What you suggested in your comment is the only general solution to the problem.

Start by trying to use the real thing (numerically if you have to) and if after that you still feel like you need a computationally lighter approximation, see how the approximation performs in the job it is intended to perform.

That's the only way to tell whether the inaccuracies of the approximation matter.

Note that for most statistical problems, the job to be performed is some sort of decision that needs to be made. So the job to be performed is rarely "what does the distribution look like" but rather "what leads to the best decision in the situation at hand?"

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.