7
$\begingroup$

I've got a stochastic variable $X$ which takes the values in $\{0,1,2\}$ with some unknown probabilities. I want to know the distribution, and can sample $X$ as many times as I want to. How many times will I need to sample to get some specific confidence interval?

(For instance, 99% sure that the estimated probabilities lie within 0.01 of the true probabilities.)

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ I think this is more related to the sample size required to obtain certain error in the estimation of the parameters of a multinomial distribution. Take a look at this link. What you can do is to narrow the C.I. by increasing the sample size. $\endgroup$ – user10525 May 23 '12 at 13:16
  • 3
    $\begingroup$ For an example of computing standard errors of estimates for linear functions of a trinomially distributed variable, please see stats.stackexchange.com/q/18603. Although the reply there does not fully answer your question, it exemplifies techniques useful for obtaining an answer. $\endgroup$ – whuber♦ May 23 '12 at 13:46
5
$\begingroup$

If you have a Bernoulli variable (0 or 1) with probability $p$, its variance is $p(1-p)$ which is always less than or equal to $1/4$. The mean of $n$ independent Bernoulli variables tends to a Gaussian of variance $p(1-p)/n$ which is always less than or equal to $1/4n$. You can use the fact that a Gaussian variable has a 99% chance of being within plus or minus 2.58 standard deviations, so you have to set your upper bound so that $2.58/(2\sqrt{n}) < .01$ which gives you $n \geq 16641$.

Because each of your three outcomes individually behaves as a Bernoulli variable and because this is a global upper bound you can also apply this number to your discrete variable with three outcomes.

Improve this answer
$\endgroup$
7
  • $\begingroup$ Does this work even though the three outcomes are mutually exclusive, i.e. not independent Bernoulli variables? $\endgroup$ – Anna May 23 '12 at 13:02
  • $\begingroup$ Yes because we took the upper bound of the variance in the first place. $\endgroup$ – gui11aume May 23 '12 at 13:08
  • $\begingroup$ +1. This certainly works but can you think of a way to give sharp confidence intervals for each proportion? That would require accounting for the dependencies between the two intervals. $\endgroup$ – Macro May 23 '12 at 13:16
  • $\begingroup$ Thanks! I made heavy use of the "and can sample X as many times as I want to" so this solution is far from optimal as you noticed. Getting optimal (sharper) confidence regions is a bit more involved I think. You could use convergence to a 3D Gaussian with terms of correlation that depend on (p,q,r) and find a region of that space that has 99% probability and that again depends on (p,q,r). Possible but the answer will depend on (p,q,r). $\endgroup$ – gui11aume May 23 '12 at 13:23
  • 2
    $\begingroup$ @gui11aume, getting sharp confidence intervals is possible as long as you don't make an approximation to Gaussian (see my answer). The problem with approximating as a Gaussian is that the approximation is only good when $n$ is very large, but that's one of the things we can't assume since we want to know how big $n$ needs to be get a good confidence interval. $\endgroup$ – Neil G May 23 '12 at 21:43
2
$\begingroup$

The Bayesian way of doing this loses no information:

Your variable $X$ is categorically distributed with probability vector $\mathbf p$. The conjugate prior of the categorical distribution is the Dirichlet distribution, so let $\mathbf p$ be Dirichlet-distributed with shape parameter vector $\boldsymbol\phi$. With every observation, you update $\boldsymbol\phi$ by incrementing the realized component. You can then check to see if your maximum likelihood probability $\mathbf p^\star$ is within 0.01 of the true probability by integrating the ball with radius 0.01 centered at $\mathbf p^\star$.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Could you explain the last sentence in a bit more detail. What is the true probability here if only the datasample are known? Integrating the ball radius, by Monte Carlo? $\endgroup$ – Vass Apr 9 '14 at 15:22
  • $\begingroup$ @Vass: The data induces a likelihood. So, the likelihood is also known. It has Dirichlet density, which you integrate to give the desired probability. $\endgroup$ – Neil G Apr 9 '14 at 20:20

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.