1
$\begingroup$

i have a data set that is being generate by a Bernoulli distribution let say $\mathbf{X} \sim x \in \{ 0,1 \}$, where x=1 with probability p. I dont have any information what is the 'p' parameter of this distribution and i am gathering sample to estimate it.

My goal is to find the best estimation for 'p' in a sample of N measures.

I want to prove that the best estimation after N measures is $f_p = \sum_{i=1}^{N} \frac{x_i}{N}$

Also would be nice to know how many sample i should have in order to estimate p with an error x over a confidence interval C

Any references that could guide me?

$\endgroup$
7
  • 1
    $\begingroup$ “Best” in terms of what exactly? Have you heard of “maximum likelihood estimation” and the properties of those estimators? $\endgroup$ Commented May 3, 2023 at 22:43
  • $\begingroup$ I second the call for clarification on what "best" means. For instance, I think there is a strong argument that the best estimator is to luck into guessing the true parameter value. $\endgroup$
    – Dave
    Commented May 3, 2023 at 22:51
  • $\begingroup$ yes, best is the estimation for the most probable value of p $\endgroup$
    – Riga
    Commented May 3, 2023 at 22:58
  • 1
    $\begingroup$ So do you just mean to ask how to perform maximum likelihood estimation? $\endgroup$
    – Dave
    Commented May 3, 2023 at 23:20
  • 3
    $\begingroup$ Wikipedia has an article binomial proportion confidence interval $\endgroup$
    – Henry
    Commented May 4, 2023 at 0:09

1 Answer 1

1
$\begingroup$

My goal is to find the best estimation for 'p' in a sample of N measures.

You haven't defined "best", but I'm willing to bet that one of the properties of the Maximum Likelihood Estimators might fit whatever description you're thinking of. When the likelihood is well behaved, as it would be here, estimates are consistent, efficient, and are asymptotically normal. In some cases, they can also be unbiased.

I want to prove that the best estimation after N measures is

This is a very simple proof and you can google something like "Maximum Likelihood Estimate of Binomial Distribution" or search this website for something similar.

Also would be nice to know how many sample i should have in order to estimate p with an error x over a confidence interval C

There are a lot of confidence intervals for the binomial proportion. I will use the simplest one for now, the Wald Interval, in the hopes you can extend the methodology to a better interval of your choosing.

The radius of the interval (what you call $x$ in your comment) is

$$ x = z_{1-\alpha/2}\sqrt{\dfrac{p(1-p)}{n}} $$

When using a 95% CI, $z_{1-\alpha/2} \approx 1.96 \approx 2$ for economy of thought. We can very easily solve for $n$

$$ n = \dfrac{4p(1-p)}{x^2} $$

We can further simplify this. Note that the variance of the binomial is bounded by 0.25, and that the variance appears in our expression. This results in

$$ n = \dfrac{4}{x^2}p(1-p) \leq \dfrac{4}{x^2}0.25 = \dfrac{1}{x^2} $$

So, in order to guarantee that the resulting confidence interval (regardless of the value of $p$) has a radius smaller than $x$, you need $1/x^2$ samples. Depending on the value of $p$, you might actually need a lot more. But this will guarantee a radius of $x$.

$\endgroup$
3
  • $\begingroup$ so thank you for respond my first question, it helped a lot $\endgroup$
    – Riga
    Commented May 4, 2023 at 15:28
  • $\begingroup$ the second one, i meant to say: how many measures i need to make in order to estimate p with an error x i.e. $p \in (p^*-x , p^*+x)$ with a well know confidence interval $\endgroup$
    – Riga
    Commented May 4, 2023 at 15:29
  • $\begingroup$ @Riga See my edited comment $\endgroup$ Commented May 4, 2023 at 17:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.