2
$\begingroup$

I am stuck at the following exercise:

Let $(X_k)_k$ be a sequence of i.i.d. Bernoulli$(p)$ distributed RVs with $p=1/2$. We want to estimate $Var(X_1) = p(1-p) = p^2 =: \tau_p^2$ by directly plugging in the natural estimator $\widetilde{p} := \frac{1}{n}\sum_{i=1}^n X_i$ of $p$. We call this estimator for the variance $\widetilde{\tau}^2_{p,n}$. Examine the limit distribution of $\widetilde{\tau}^2_{p,n}$.

I do not see how I could do this. I recognise that we are basically looking at a uniform distribution of two events and I see that $\widetilde{\tau}^2_{p,n}$ has a rather nice formula

$$\widetilde{\tau}^2_{p,n} = \frac{1}{n^2} \biggl(\sum_{i=1}^n X_i \biggr) ^2 = \overline{X}^2,$$

but I do not see how to proceed now. Intuitively I would say that $\widetilde{\tau}^2_{p,n}$ should also be uniformly distributed, but how can I prove this?

$\endgroup$
2
  • $\begingroup$ To see that your intuition is incorrect, consider that the distribution of $\tilde p$ must be $1/n$ times a Binomial$(n,p)$ distribution, which for $n\gt 1$ is far from uniform, suggesting (correctly) the distribution of $\tilde{\tau}^2_{p,n}$ is nonuniform, too. BTW, the limit distribution of $\tilde{\tau}^2_{p,n}$ is the constant $1/4$ -- but that's a trivial and almost worthless observation. The question likely wants you to standardize the distribution. $\endgroup$
    – whuber
    Nov 10, 2020 at 22:57
  • $\begingroup$ Could you use a Monte Carlo simulation? $\endgroup$ Nov 10, 2020 at 23:03

1 Answer 1

4
$\begingroup$

Firstly, let's start by fixing your estimator. Although $p(1-p)=p^2$ in the special case where $p=\tfrac{1}{2}$, in the context of parameter estimation, we don't know the parameter $p$, and so we don't know that $p(1-p)=p^2$. Consequently, the plug-in variance estimator is $\tilde{\tau}^2_{n} = \tilde{p}_n (1-\tilde{p}_n)$, not $\tilde{p}_n^2$. I will show an asymptotic approximation derived using the central limit theorem to approximate the true distribution function for the estimator. This is quite a tricky problem, and it has a few parts, but it leads to quite a useful asymptotic form.


Let's start by writing the variance estimator out in terms of the number of "successes" in the underlying Bernoulli random variables. Let $K_n \equiv \sum_{i=1}^n X_i$ denote the number of successes. Then we can write the plug-in estimator as:

$$\tilde{\tau}^2_{n} = \tilde{p}_n (1-\tilde{p}_n) = \frac{1}{n^2} (n K_n - K_n^2).$$

We will also note that, in the present case where $p=\tfrac{1}{2}$, we can use the central limit theorem to obtain the asymptotic distribution:

$$Z_n \equiv 2 \sqrt{n} \Bigg( \frac{K_n}{n} - \frac{1}{2} \Bigg) \sim \text{N}(0,1).$$

The range of possible values for the variance estimator is $0 \leqslant \tilde{\tau}^2_{n} \leqslant \tfrac{1}{4}$. For all $0 \leqslant t \leqslant \tfrac{1}{4}$, the cumulative distribution function for this estimator is given by:

$$\begin{align} \mathbb{P}(\tilde{\tau}^2_{n} \leqslant t) &= \mathbb{P}(n K_n - K_n^2 \leqslant t n^2) \\[14pt] &= \mathbb{P}(K_n^2 - n K_n + t n^2 \geqslant 0) \\[8pt] &= 1 - \mathbb{P} \Bigg( \frac{1 - \sqrt{1-4t}}{2} < \frac{K_n}{n} < \frac{1 + \sqrt{1-4t}}{2} \Bigg) \\[6pt] &= 1 - \mathbb{P} \Bigg( - \frac{\sqrt{1-4t}}{2} < \frac{K_n}{n} - \frac{1}{2} < \frac{\sqrt{1-4t}}{2} \Bigg) \\[6pt] &= 1 - \mathbb{P} \Bigg( - \sqrt{(1-4t)n} < 2 \sqrt{n} \Bigg( \frac{K_n}{n} - \frac{1}{2} \Bigg) < \sqrt{(1-4t)n} \Bigg) \\[8pt] &= 1 - \mathbb{P} \Big( - \sqrt{(1-4t)n} < Z_n < \sqrt{(1-4t)n} \Big). \\[6pt] \end{align}$$

(The third step in this equation follows from re-arrangement of the quadratic inequality; full working shown below.) When $n$ is large we have the asymptotic approximation $Z_n \sim \text{N}(0,1)$ so we get the asymptotic approximation $\mathbb{P}(\tilde{\tau}^2_{n} \leqslant t) \approx F_\text{asymp}(t)$ given by:

$$F_\text{asymp}(t) = 1 - \frac{\Phi ( \sqrt{(1-4t)n} ) - \Phi ( -\sqrt{(1-4t)n} )}{\Phi (\sqrt{n}) - \Phi (-\sqrt{n})}.$$

(Note that I have added a denominator here to obtain a distribution function that is valid for all $n$ over the range $0 \leqslant t \leqslant \tfrac{1}{4}$. This form ensures that we get zero probability when $t=0$ and unit probability when $t = \tfrac{1}{4}$.) Differentiating then gives the asymptotic density:

$$\begin{align} f_\text{asymp}(t) &= \sqrt{\frac{n}{2 \pi (1-4t)}} \cdot \frac{\exp ( - (1-4t)n/2 )}{\Phi (\sqrt{n}) - \Phi (-\sqrt{n})} \cdot \mathbb{I}(0 \leqslant t \leqslant \tfrac{1}{4}) \\[6pt] &= \frac{\mathbb{I}(0 \leqslant 1-4t \leqslant 1)}{\Phi (\sqrt{n}) - \Phi (-\sqrt{n})} \cdot \text{Ga} \Big( 1-4t \ \Big| \ \text{Shape} = \frac{1}{2}, \ \text{Rate} = \frac{n}{2} \Big). \\[6pt] \end{align}$$

This means that our estimator has an asymptotic distribution that is a truncated gamma distribution:

$$1 - 4 \tilde{\tau}^2_{n} \overset{\text{Asymp}}{\sim} \text{TruncGamma} \Big( \text{Shape} = \frac{1}{2}, \ \text{Rate} = \frac{n}{2}, \ \text{Upper Bound} = 1 \Big).$$

This gives you an asymptotic density function based on approximating the binomial random variable $K_n$ using the central limit theorem. The asymptotic form is a continuous approximation to a discrete distribution, but it should give quite a good approximation to the true distribution when $n$ is large.


Working for the quadratic inequality: Using quadratic formula we can find the roots of the quadratic form shown in the inequality, which are:

$$r = n \cdot \frac{1 \pm \sqrt{1-4t}}{2}.$$

Consequently, we can write the quadratic form as:

$$K_n^2 - n K_n + t n^2 = \Big(K_n - n \cdot \frac{1 - \sqrt{1-4t}}{2} \Big) \Big( K_n - n \cdot \frac{1 + \sqrt{1-4t}}{2} \Big).$$

The quadratic inequality $K_n^2 - n K_n + t n^2 \geqslant 0$ occurs when both terms are non-negative, or both terms are non-positive. This excludes the range:

$$\frac{1 - \sqrt{1-4t}}{2} < \frac{K_n}{n} < \frac{1 + \sqrt{1-4t}}{2}.$$

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.