12
$\begingroup$

How do we define an estimator for data coming from a binomial distribution? For bernoulli I can think of an estimator estimating a parameter p, but for binomial I can't see what parameters to estimate when we have n characterizing the distribution?

Update:

By an estimator I mean a function of the observed data. An estimator is used to estimate the parameters of the distribution generating the data.

$\endgroup$
  • $\begingroup$ What is your understanding of an "estimator"? I wonder about that, because estimators don't have "parameters." It makes me concerned that you aren't clearly communicating your question. Maybe you could give a concrete example of an actual situation you're considering. $\endgroup$ – whuber Oct 7 '11 at 18:41
  • $\begingroup$ @whuber added more information. let me know if you want me to add more details or if my understanding is flawed. $\endgroup$ – Rohit Banga Oct 7 '11 at 19:04
  • $\begingroup$ The edit is correct, but a concrete example would still help. In many applications of the Binomial distribution, $n$ is not a parameter: it is given and $p$ is the only parameter to be estimated. For example, the count $k$ of successes in $n$ independent identically distributed Bernoulli trials has a Binomial($n$,$p$) distribution and one estimator of the sole parameter $p$ is $k/n$. $\endgroup$ – whuber Oct 7 '11 at 19:36
  • 2
    $\begingroup$ I would love to see an example, even a contrived one, of estimating both $n$ and $p$ (in a frequentist setting). Think about it: you observe a single count, k, say $k=5$. We expect $k$ approximately to equal $n p$. So do we estimate $n=10$, $p=0.5$? Or maybe $n=5000$, $p=0.001$? Or almost anything else? :-) Or are you suggesting you might have a series of independent observations $k_1, k_2, \ldots, k_m$ all from a common Binomial$(n,p)$ distribution with both $p$ and $n$ unknown? $\endgroup$ – whuber Oct 7 '11 at 22:19
  • 1
    $\begingroup$ I am suggesting the latter - both p and n are unknown. I want an estimator for both n and p as a function of N observed data points. $\endgroup$ – Rohit Banga Oct 7 '11 at 23:34
12
$\begingroup$

I guess what you are looking for is the probability generating function. A derivation of the probability generating function of the binomial distribution can be found under

http://economictheoryblog.com/2012/10/21/binomial-distribution/

However, having a look at Wikipedia is nowadays always a good idea, although I have to say that the specification of the binomial could be improved.

https://en.wikipedia.org/wiki/Binomial_distribution#Specification

$\endgroup$
1
$\begingroup$

Every distribution have some unknown parameter(s). For example in the Bernoulli distribution has one unknown parameter probability of success (p). Likewise in the Binomial distribution has two unknown parameters n and p. It depends on your objective which unknown parameter you want to estimate. you can fix one parameter and estimation other one. For more information see this

$\endgroup$
  • $\begingroup$ What if I want to estimate both the parameters? $\endgroup$ – Rohit Banga Oct 7 '11 at 23:59
  • 1
    $\begingroup$ For maximum likelihood estimation, you have to take derivative of the likelihood function with respect to interested parameter(s) and equate that equation to zero, and solve the equation. I mean to say the procedure is same as you did while estimating 'p'. You have to do same with 'n'. check this one www.montana.edu/rotella/502/binom_like.pdf $\endgroup$ – love-stats Oct 8 '11 at 0:35
  • $\begingroup$ @love Your reference estimates only $p$, taking $N$ as fixed. $\endgroup$ – whuber Oct 10 '11 at 15:04
  • $\begingroup$ -1 @love-stats For an example of a situation where taking the derivative of the likelihood function, equating it to $0$, etc. does not work, see this attempt and the correct solution $\endgroup$ – Dilip Sarwate Nov 2 '12 at 11:52
1
$\begingroup$

Say you have data $k_1, \dots, k_m \sim \text{iid binomial}(n, p)$.

You could easily derive method-of-moment estimators by setting $\bar{k} = \hat{n}\hat{p}$ and $s_k^2 = \hat{n}\hat{p}(1-\hat{p})$ and solving for $\hat{n}$ and $\hat{p}$.

Or you could calculate MLEs (perhaps just numerically), eg using optim in R.

$\endgroup$
  • $\begingroup$ It turns out the MLEs are really horrible for $p \lt 1/2$--they are biased and hugely variable, even with large samples. I haven't studied the MM estimators, in part because they're frequently not even defined (whenever $s^2/\bar{k} \gt 1$, which happens). $\endgroup$ – whuber Oct 11 '11 at 20:35
  • $\begingroup$ @whuber - he didn't ask for a good estimator. ;) $\endgroup$ – Karl Oct 11 '11 at 20:40
  • 1
    $\begingroup$ Why not just propose $\hat{n}$=17 and $\hat{p}=1/2$ no matter what, then? :-) But you have a point: the question doesn't even specify what is to be estimated. If we only need an estimator for $np$, then there's an obvious good one available. $\endgroup$ – whuber Oct 11 '11 at 20:43
  • $\begingroup$ @whuber - Indeed. And I wouldn't be surprised to find $\hat{n} \approx \max k_i$ for the MLE. $\endgroup$ – Karl Oct 11 '11 at 20:48
  • $\begingroup$ That's correct: especially when $p$ is close to $1$, the max of the counts is the MLE. It works pretty well in such cases, as you might imagine. For smaller $p$, even with lots of data it's hard to distinguish this from a Poisson distribution, for which $n$ is effectively infinite, leading to an enormous uncertainty in the estimate of $n$. $\endgroup$ – whuber Oct 11 '11 at 21:40
0
$\begingroup$

I think we could use method of moments estimation to estimate the parameters of the Binomial distribution by the mean and the variance.


Using the method of moments estimation to estimate The parameters $p$ and $m$. [{\hat{p}}_n=\frac{\overline{X}-S^2}{\overline{X}}][\hat{m}_n=\frac{\overline{X}^2}{\overline{X}-S^2}] Proof The estimators of the parameters $m$ and $p$ by the Method of Moments are the solutions of the system of equations $$mp =\bar{X},\quad mp(1-p) = S^2.$$ Hence our equations for the method of moments are: [\overline{X}=mp] [S^2=mp (1-p).]

Simple arithmetic shows: [S^2 = mp\left(1 - p\right) = \bar{X}\left(1 - p\right)] [S^2=\bar{X}-\bar{X} p] [\bar{X}p=\bar{X}-S^2, \mbox{ therefore } \hat{p}=\frac{\bar{X}-S^2}{\bar{X}}.] Then, [\bar{X} = mp, \mbox{ that is, } m \left(\frac{\bar{X}-S^2}{\bar{X}}\right)] [\bar{X}=m\left(\frac{\bar{X}-S^2}{\bar{X}}\right), \mbox{ or } \hat{m}=\frac{\bar{X}^2}{\bar{X}-S^2}. ]

$\endgroup$
  • 1
    $\begingroup$ It would be good if you could expand on this, for example, by writing the formula for the MoM estimator. Otherwise the answer is not self-contained; others (who don't already know the answer) will have to search online for "method of moments" etc. until they find the real answer. $\endgroup$ – jbowman Mar 11 at 20:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.