Given $m$ i.i.d. Bernoulli( $\theta$ ) r.v.s $X_{1}, X_{2}, \ldots, X_{m},$ I'm interested in finding the UMVUE of $(1-\theta)^{1/k}$, when $k$ is a positive integer. .

I know $\sum X_{i}$ is a sufficient statistic by the Factorization Theorem, but I'm having trouble proceeding from there. If I can find an unbiased function of the sufficient statistic the problem is solved by the Rao-Blackwell theorem.

  • $\begingroup$ @StubbornAtom , I am confused about the infinite series in your comment. what is $nchoosek(m,j)$ when m is a fraction. Did you assume $m$ is an integer? $\endgroup$ – wanderer Apr 20 at 15:24
  • $\begingroup$ No. It is a general formula: en.wikipedia.org/wiki/Binomial_series $\endgroup$ – StubbornAtom Apr 20 at 15:28
  • $\begingroup$ @Xi'an, This is the first time I am hearing about "Bernoulli Factory solutions". Could you elaborate how I can use the new random variable with parameter $\Theta^{a}$ to estimate $\Theta^{a}$. Are you suggesting to generate this new RV and use its sample mean as the setimate? $\endgroup$ – wanderer Apr 20 at 18:31

Except when $k=1$, given a finite sequence of i.i.d. Bernoulli $\mathcal B(θ)$ random variables $X_1,X_2,\ldots,X_m$, there exists no unbiased estimator of $(1−θ)^{1/k}$, when $k$ is a positive integer.

The reason for this impossibility is that only polynomials in $\theta$ of degree at most $m$ can be unbiasedly estimated. Indeed, since $Y_m=m\bar{X}_m$ is a sufficient statistic, we can assume wlog that an unbiased estimator is a function of $Y_m\sim\mathcal Bin(m,p)$, $\delta(Y_m)$, with expectation $$\sum_{i=0}^m \delta(i) {m \choose i} \theta^i(1-\theta)^{m-i}$$ which is therefore a polynomial in $\theta$ of degree at most $m$.

See Halmos (1946) for a general theory of unbiased estimation that points out the rarity of unbiasedly estimable functions.

However, when changing the perspective, there exists an unbiased estimator of $\theta^a$, $a\in(0,1)$, when considering instead an infinite sequence of i.i.d. Bernoulli $\mathcal B(θ)$ random variables $X_1,X_2,\ldots$ This is a consequence of the notion of a Bernoulli factory.

Given a known function $f:S\mapsto (0,1)$, we consider the problem of using independent tosses of a coin with probability of heads $\theta$ (where $\theta\in S$ is unknown) to simulate a coin with probability of heads $f(\theta)$. (Nacu & Peres, 2005)

Mendo (2018) and Thomas and Blanchet show that there exists a Bernoulli factory solution for $θ^a$, $a\in (0,1)$, with constructive arguments. The first author uses the power series decomposition of $f(\theta)$ $$f(\theta)=1-\sum_{k=1}^\infty c_k(1-\theta)^k\qquad c_k\ge 0,\,\sum_{k=1}^\infty c_k=1$$ to construct the sequence$$d_k=\dfrac{c_k}{1-\sum_{\kappa=1}^{k-1}c_\kappa}$$ and the associated algorithm

  1. Set i=1.
  2. Take one Bernoulli $\mathcal B(θ)$ input Xi.
  3. Produce Ui Uniform on (0,1). Let Vi = 1 if Ui < di or Vi = 0 otherwise.
  4. If Vi or Xi are 1, output Y = Xi and finish. Else increase i by 1 and go back to step 2.

    For instance, when $f(\theta) =\sqrt\theta$ the coefficients $c_k$ are $$c_k=\frac{1}{2^{2k−1}k}{2k-2 \choose k−1}$$ Here is an R code illustrating the validity of the method:

    ck=exp(lchoose(n=2*(k<-1:1e5)-2,k=k-1)-log(k)-{2*k-1}*log(2)) dk=ck/(1-c(0,cumsum(ck[-1e5]))) be <- function(p){ i=1 while((xi<-runif(1)>p)&(runif(1)>dk[i])) i=i+1 1-xi} for(t in 1:1e5)ck[t]=be(p)

and the empirical verification that the simulated outcomes are indeed Bernoulli $\mathcal B(\sqrt{\theta})$:

enter image description here

As an aside estimating $\theta^{1/k}$ or $(1-\theta)^{1/k}$ has a practical appeal when considering Dorfman's group blood testing or pooling where blood samples of $k$ individuals are mixed together to speed up the confirmation they all are free from a disease.

| cite | improve this answer | |
  • $\begingroup$ @Xian, taking a second look at this answer, I feels like we are here proving the non-existence of an unbiased estimator. That can be useful to argue that no estimator achieves CRLB. But still an MVUE can exist right? $\endgroup$ – wanderer Apr 25 at 6:34
  • $\begingroup$ The "U" in MVUE stands for unbiased. $\endgroup$ – Xi'an Apr 25 at 10:55

First of all, I'll just point out that it's not enough that $\sum_i X_i$ is sufficient. We need it to be complete-sufficient. Fortunately we know that $\sum_i X_i$ is also a complete statistic by well known properties of the exponential family of distributions.

As you say, we need an estimator $\delta(\cdot)$ based on the complete-sufficient statistic $T(X)$ which is unbiased, i.e. we need $$\mathbb{E} [\delta (T(X)) ] = (1-\theta)^{1/k}.$$ One approach to solving this problem is to solve for the function $\delta(\cdot)$.

We know that $\sum_i X_i \sim Binomial(m,\theta)$. Thus, we can write out the expected value of $\delta (\sum_i X_i)$ as:

$$\mathbb{E} [\delta (\sum_i X_i) ] = \sum_{k=0}^m \delta (k) {m \choose k} \theta^k (1-\theta)^{n-k}$$

We want the right hand side to equal $(1-\theta)^{1/k}$, so that $\delta (\cdot)$ is unbiased and hence UMVUE. Hence, you need to solve the following for $\delta (\cdot)$:

$$ \sum_{k=0}^n \delta (k) {m \choose k} \theta^k (1-\theta)^{m-k} = (1-\theta)^{1/k}\tag{1}$$

The answer is not immediately obvious to me, but this is one of two standard approaches when deriving UMVUE's. The other approach is to start with any unbiased estimator and condition on a complete sufficient statistic.

For example, suppose you know that there is an estimator $g(\cdot)$ such that $E[g(\vec{X})] = (1-\theta)^{1/k}$, so that it is unbiased but not UMVUE. Then it follows that $\delta (\sum_i X_i) = E[g(\vec{X})|\sum_i X_i]$ is UMVUE.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.