# Unbiased estimator of binomial PMF

Is there an unbiased estimator of PMF of a random variable $$Y=\sum_{i=1}^{n} X_n$$ where $$X_i$$ are independent Bernoulli trials with probability $$p$$, that is, the estimator of: $$\begin{equation}\tag{1} f(k,n)=P(Y=k|n)={n\choose k}p^k(1-p)^{n-k} \end{equation}$$ for an arbitrary $$n \geq 0$$ and $$n\geq k \geq 0$$ when based on a sample of $$s$$ observations from that said Bernoulli RV; $$\lbrace x_i\rbrace_{i=1}^{s}$$?

More info and progress so far:
An obvious candidate for the estimator of PMF would be: $$\begin{equation}\tag{2} \hat{f}(k,n)={n\choose k}\hat{p}^k(1-\hat{p})^{n-k} \end{equation}$$ where $$\hat{p}= \frac{\sum_{i=1}^{s}x_i}{s}$$. That is consistent but clearly not unbiased as apparent from Jensen's inequality. So far I found that the estimator based on hypergeometric distribution: $$\begin{equation}\tag{3} \hat{f}(k,n)=\dfrac{{\hat{p}*s\choose k} {s-\hat{p}*s\choose n-k}}{{n\choose k}} \end{equation}$$ with $$\hat{p}$$ defined as above seems to be unbiased (at least in my simulations, although I wasn't able to prove it). This function $$\hat{f}(k,n)$$ is however usefull only for $$n\leq s$$. Does there exist some unbiased estimator of $$f(k,n)$$ which would work also for $$n>s$$? (Possibly some correction of estimator (2) or some generalization of estimator (3). Or something completely different.)

## 1 Answer

Since, for a Binomial $$\text{B}(n,p)$$ variable $$X$$, and $$k\le n$$, the factorial moment is given by $$\mathbb{E}_p[X(X-1)\cdots(X-k+1)] = n(n-1)\cdots(n-k+1)p^k,$$ the $$s$$ Bernoulli rvs $$\lbrace X_i\rbrace_{i=1}^{s}$$ can easily return independent unbiased estimates of both $$p^k$$ and $$(1-p)^{n-k}$$ if $$k+(n-k)\le s$$, that is, if $$n\le s$$. And hence of $$\mathbb{P}(X=k)\propto p^k(1-p)^{n-k}$$.

It sounds likely that an unbiased estimator of the above does not exist when $$n>s$$, because, for instance, developing $$(X_1+\ldots+X_s)^k$$ shows that the maximum number of terms in a product is $$s$$, with expectation $$p^s$$. No higher power of $$p$$ or $$(1-p)$$ can appear for this reason. Actually, the proof is straightforward: consider there exists such an unbiased estimator, denoted by $$G(X_1+\ldots+X_s)$$ since by sufficiency there exists an unbiased estimator based on the sum. Then it satisfies $$\mathbb{E}_p[G(X_1+\ldots+X_s)]=\sum_{j=1}^s \underbrace{G(j){s \choose j}}_\text{independent from p}p^j(1-p)^{s-j}=p^k(1-p)^{n-k}$$ or $$\sum_{j=1}^s \overbrace{G(j){s \choose j}}^{\text{non-negative}}p^{j-k}(1-p)^{s-j-n+k}=1$$ Letting $$p$$ tend to $$0$$ or $$1$$ leads to explosive terms when $$j-k<0$$ and when $$s-j-n+k<0$$, unless the coefficient $$G(j){s \choose j}p^{j-k}$$ is equal to zero. If $$n>s$$, then, for all $$0\le j\le s$$ and all $$0\le k\le n$$, either $$j or $$s-j, which leads to an impossibility since $$G(m)=0$$ for all $$0\le m\le s$$. Therefore there is no unbiased estimator of $$p^k(1-p)^{n-k}$$ when $$n>s$$.

• Thank you very much, professor Xi'an. That is a very neat counterexample. Now I know that my search for such estimator is futile and can allocate my time accordingly. Thanks again. – Nicolle Jan 6 '19 at 19:07
• Thank you, this is a very neat case about the rarity of functions that allow for an unbiased estimator, a rarity that is often forgotten by textbooks. – Xi'an Jan 6 '19 at 19:50