16
$\begingroup$

How do we define an estimator for data coming from a binomial distribution? For bernoulli I can think of an estimator estimating a parameter p, but for binomial I can't see what parameters to estimate when we have n characterizing the distribution?

Update:

By an estimator I mean a function of the observed data. An estimator is used to estimate the parameters of the distribution generating the data.

$\endgroup$
6
  • 1
    $\begingroup$ What is your understanding of an "estimator"? I wonder about that, because estimators don't have "parameters." It makes me concerned that you aren't clearly communicating your question. Maybe you could give a concrete example of an actual situation you're considering. $\endgroup$
    – whuber
    Oct 7, 2011 at 18:41
  • $\begingroup$ @whuber added more information. let me know if you want me to add more details or if my understanding is flawed. $\endgroup$ Oct 7, 2011 at 19:04
  • $\begingroup$ The edit is correct, but a concrete example would still help. In many applications of the Binomial distribution, $n$ is not a parameter: it is given and $p$ is the only parameter to be estimated. For example, the count $k$ of successes in $n$ independent identically distributed Bernoulli trials has a Binomial($n$,$p$) distribution and one estimator of the sole parameter $p$ is $k/n$. $\endgroup$
    – whuber
    Oct 7, 2011 at 19:36
  • 2
    $\begingroup$ I would love to see an example, even a contrived one, of estimating both $n$ and $p$ (in a frequentist setting). Think about it: you observe a single count, k, say $k=5$. We expect $k$ approximately to equal $n p$. So do we estimate $n=10$, $p=0.5$? Or maybe $n=5000$, $p=0.001$? Or almost anything else? :-) Or are you suggesting you might have a series of independent observations $k_1, k_2, \ldots, k_m$ all from a common Binomial$(n,p)$ distribution with both $p$ and $n$ unknown? $\endgroup$
    – whuber
    Oct 7, 2011 at 22:19
  • 1
    $\begingroup$ I am suggesting the latter - both p and n are unknown. I want an estimator for both n and p as a function of N observed data points. $\endgroup$ Oct 7, 2011 at 23:34

4 Answers 4

12
$\begingroup$

I guess what you are looking for is the probability generating function. A derivation of the probability generating function of the binomial distribution can be found under

http://economictheoryblog.com/2012/10/21/binomial-distribution/

However, having a look at Wikipedia is nowadays always a good idea, although I have to say that the specification of the binomial could be improved.

https://en.wikipedia.org/wiki/Binomial_distribution#Specification

$\endgroup$
3
$\begingroup$

Say you have data $k_1, \dots, k_m \sim \text{iid binomial}(n, p)$.

You could easily derive method-of-moment estimators by setting $\bar{k} = \hat{n}\hat{p}$ and $s_k^2 = \hat{n}\hat{p}(1-\hat{p})$ and solving for $\hat{n}$ and $\hat{p}$.

Or you could calculate MLEs (perhaps just numerically), eg using optim in R.

$\endgroup$
6
  • $\begingroup$ It turns out the MLEs are really horrible for $p \lt 1/2$--they are biased and hugely variable, even with large samples. I haven't studied the MM estimators, in part because they're frequently not even defined (whenever $s^2/\bar{k} \gt 1$, which happens). $\endgroup$
    – whuber
    Oct 11, 2011 at 20:35
  • $\begingroup$ @whuber - he didn't ask for a good estimator. ;) $\endgroup$
    – Karl
    Oct 11, 2011 at 20:40
  • 1
    $\begingroup$ Why not just propose $\hat{n}$=17 and $\hat{p}=1/2$ no matter what, then? :-) But you have a point: the question doesn't even specify what is to be estimated. If we only need an estimator for $np$, then there's an obvious good one available. $\endgroup$
    – whuber
    Oct 11, 2011 at 20:43
  • $\begingroup$ @whuber - Indeed. And I wouldn't be surprised to find $\hat{n} \approx \max k_i$ for the MLE. $\endgroup$
    – Karl
    Oct 11, 2011 at 20:48
  • $\begingroup$ That's correct: especially when $p$ is close to $1$, the max of the counts is the MLE. It works pretty well in such cases, as you might imagine. For smaller $p$, even with lots of data it's hard to distinguish this from a Poisson distribution, for which $n$ is effectively infinite, leading to an enormous uncertainty in the estimate of $n$. $\endgroup$
    – whuber
    Oct 11, 2011 at 21:40
1
$\begingroup$

Every distribution have some unknown parameter(s). For example in the Bernoulli distribution has one unknown parameter probability of success (p). Likewise in the Binomial distribution has two unknown parameters n and p. It depends on your objective which unknown parameter you want to estimate. you can fix one parameter and estimation other one. For more information see this

$\endgroup$
4
  • $\begingroup$ What if I want to estimate both the parameters? $\endgroup$ Oct 7, 2011 at 23:59
  • 1
    $\begingroup$ For maximum likelihood estimation, you have to take derivative of the likelihood function with respect to interested parameter(s) and equate that equation to zero, and solve the equation. I mean to say the procedure is same as you did while estimating 'p'. You have to do same with 'n'. check this one www.montana.edu/rotella/502/binom_like.pdf $\endgroup$
    – love-stats
    Oct 8, 2011 at 0:35
  • $\begingroup$ @love Your reference estimates only $p$, taking $N$ as fixed. $\endgroup$
    – whuber
    Oct 10, 2011 at 15:04
  • $\begingroup$ -1 @love-stats For an example of a situation where taking the derivative of the likelihood function, equating it to $0$, etc. does not work, see this attempt and the correct solution $\endgroup$ Nov 2, 2012 at 11:52
1
$\begingroup$

I think we could use method of moments estimation to estimate the parameters of the Binomial distribution by the mean and the variance.


Using the method of moments estimation to estimate The parameters $p$ and $m$. [{\hat{p}}_n=\frac{\overline{X}-S^2}{\overline{X}}][\hat{m}_n=\frac{\overline{X}^2}{\overline{X}-S^2}] Proof The estimators of the parameters $m$ and $p$ by the Method of Moments are the solutions of the system of equations $$mp =\bar{X},\quad mp(1-p) = S^2.$$ Hence our equations for the method of moments are: [\overline{X}=mp] [S^2=mp (1-p).]

Simple arithmetic shows: [S^2 = mp\left(1 - p\right) = \bar{X}\left(1 - p\right)] [S^2=\bar{X}-\bar{X} p] [\bar{X}p=\bar{X}-S^2, \mbox{ therefore } \hat{p}=\frac{\bar{X}-S^2}{\bar{X}}.] Then, [\bar{X} = mp, \mbox{ that is, } m \left(\frac{\bar{X}-S^2}{\bar{X}}\right)] [\bar{X}=m\left(\frac{\bar{X}-S^2}{\bar{X}}\right), \mbox{ or } \hat{m}=\frac{\bar{X}^2}{\bar{X}-S^2}. ]

$\endgroup$
2
  • 1
    $\begingroup$ It would be good if you could expand on this, for example, by writing the formula for the MoM estimator. Otherwise the answer is not self-contained; others (who don't already know the answer) will have to search online for "method of moments" etc. until they find the real answer. $\endgroup$
    – jbowman
    Mar 11, 2019 at 20:30
  • 1
    $\begingroup$ is there a way to render the math here correctly? $\endgroup$ Feb 10, 2020 at 16:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.