Asymptotic distribution of "plug-in" variance estimator for Bernoulli data I am stuck at the following exercise:

Let $(X_k)_k$ be a sequence of i.i.d. Bernoulli$(p)$ distributed RVs with $p=1/2$. We want to estimate $Var(X_1) = p(1-p) = p^2 =: \tau_p^2$ by directly plugging in the natural estimator $\widetilde{p} := \frac{1}{n}\sum_{i=1}^n X_i$ of $p$. We call this estimator for the variance $\widetilde{\tau}^2_{p,n}$. Examine the limit distribution of $\widetilde{\tau}^2_{p,n}$.

I do not see how I could do this. I recognise that we are basically looking at a uniform distribution of two events and I see that  $\widetilde{\tau}^2_{p,n}$ has a rather nice formula
$$\widetilde{\tau}^2_{p,n} = \frac{1}{n^2} \biggl(\sum_{i=1}^n X_i \biggr) ^2 = \overline{X}^2,$$
but I do not see how to proceed now.  Intuitively I would say that $\widetilde{\tau}^2_{p,n}$ should also be uniformly distributed, but how can I prove this?
 A: Firstly, let's start by fixing your estimator.  Although $p(1-p)=p^2$ in the special case where $p=\tfrac{1}{2}$, in the context of parameter estimation, we don't know the parameter $p$, and so we don't know that $p(1-p)=p^2$.  Consequently, the plug-in variance estimator is $\tilde{\tau}^2_{n} = \tilde{p}_n (1-\tilde{p}_n)$, not $\tilde{p}_n^2$.  I will show an asymptotic approximation derived using the central limit theorem to approximate the true distribution function for the estimator.  This is quite a tricky problem, and it has a few parts, but it leads to quite a useful asymptotic form.

Let's start by writing the variance estimator out in terms of the number of "successes" in the underlying Bernoulli random variables.  Let $K_n \equiv \sum_{i=1}^n X_i$ denote the number of successes.  Then we can write the plug-in estimator as:
$$\tilde{\tau}^2_{n} 
= \tilde{p}_n (1-\tilde{p}_n) 
= \frac{1}{n^2} (n K_n - K_n^2).$$
We will also note that, in the present case where $p=\tfrac{1}{2}$, we can use the central limit theorem to obtain the asymptotic distribution:
$$Z_n \equiv 2 \sqrt{n} \Bigg( \frac{K_n}{n} - \frac{1}{2} \Bigg) \sim \text{N}(0,1).$$
The range of possible values for the variance estimator is $0 \leqslant \tilde{\tau}^2_{n} \leqslant \tfrac{1}{4}$.  For all $0 \leqslant t \leqslant \tfrac{1}{4}$, the cumulative distribution function for this estimator is given by:
$$\begin{align}
\mathbb{P}(\tilde{\tau}^2_{n} \leqslant t)
&= \mathbb{P}(n K_n - K_n^2 \leqslant t n^2) \\[14pt]
&= \mathbb{P}(K_n^2 - n K_n + t n^2 \geqslant 0) \\[8pt]
&= 1 - \mathbb{P} \Bigg( \frac{1 - \sqrt{1-4t}}{2} < \frac{K_n}{n} < \frac{1 + \sqrt{1-4t}}{2} \Bigg) \\[6pt]
&= 1 - \mathbb{P} \Bigg( - \frac{\sqrt{1-4t}}{2} < \frac{K_n}{n} - \frac{1}{2} < \frac{\sqrt{1-4t}}{2} \Bigg) \\[6pt]
&= 1 - \mathbb{P} \Bigg( - \sqrt{(1-4t)n} < 2 \sqrt{n} \Bigg( \frac{K_n}{n} - \frac{1}{2} \Bigg) < \sqrt{(1-4t)n} \Bigg) \\[8pt]
&= 1 - \mathbb{P} \Big( - \sqrt{(1-4t)n} < Z_n < \sqrt{(1-4t)n} \Big). \\[6pt]
\end{align}$$
(The third step in this equation follows from re-arrangement of the quadratic inequality; full working shown below.)  When $n$ is large we have the asymptotic approximation $Z_n \sim \text{N}(0,1)$ so we get the asymptotic approximation $\mathbb{P}(\tilde{\tau}^2_{n} \leqslant t)
\approx F_\text{asymp}(t)$ given by:
$$F_\text{asymp}(t)
= 1 - \frac{\Phi ( \sqrt{(1-4t)n} ) - \Phi ( -\sqrt{(1-4t)n} )}{\Phi (\sqrt{n}) - \Phi (-\sqrt{n})}.$$
(Note that I have added a denominator here to obtain a distribution function that is valid for all $n$ over the range $0 \leqslant t \leqslant \tfrac{1}{4}$.  This form ensures that we get zero probability when $t=0$ and unit probability when $t = \tfrac{1}{4}$.)  Differentiating then gives the asymptotic density:
$$\begin{align}
f_\text{asymp}(t) 
&= \sqrt{\frac{n}{2 \pi (1-4t)}} \cdot \frac{\exp ( - (1-4t)n/2 )}{\Phi (\sqrt{n}) - \Phi (-\sqrt{n})} \cdot \mathbb{I}(0 \leqslant t \leqslant \tfrac{1}{4}) \\[6pt]
&= \frac{\mathbb{I}(0 \leqslant 1-4t \leqslant 1)}{\Phi (\sqrt{n}) - \Phi (-\sqrt{n})} \cdot \text{Ga} \Big( 1-4t \ \Big| \ \text{Shape} = \frac{1}{2}, \ \text{Rate} = \frac{n}{2} \Big). \\[6pt]
\end{align}$$
This means that our estimator has an asymptotic distribution that is a truncated gamma distribution:
$$1 - 4 \tilde{\tau}^2_{n} \overset{\text{Asymp}}{\sim} \text{TruncGamma} \Big( \text{Shape} = \frac{1}{2}, \ \text{Rate} = \frac{n}{2}, \ \text{Upper Bound} = 1 \Big).$$
This gives you an asymptotic density function based on approximating the binomial random variable $K_n$ using the central limit theorem.  The asymptotic form is a continuous approximation to a discrete distribution, but it should give quite a good approximation to the true distribution when $n$ is large.

Working for the quadratic inequality: Using quadratic formula we can find the roots of the quadratic form shown in the inequality, which are:
$$r = n \cdot \frac{1 \pm \sqrt{1-4t}}{2}.$$
Consequently, we can write the quadratic form as:
$$K_n^2 - n K_n + t n^2 
= \Big(K_n - n \cdot \frac{1 - \sqrt{1-4t}}{2} \Big) \Big( K_n - n \cdot \frac{1 + \sqrt{1-4t}}{2} \Big).$$
The quadratic inequality $K_n^2 - n K_n + t n^2 \geqslant 0$ occurs when both terms are non-negative, or both terms are non-positive.  This excludes the range:
$$\frac{1 - \sqrt{1-4t}}{2} < \frac{K_n}{n} < \frac{1 + \sqrt{1-4t}}{2}.$$
