Let $$X_1,\dots,X_m$$ are i.i.d. with distribution function $F$ and $$Y_1,\dots,Y_n$$ are i.i.d. with distribution function $G$. Suppose that there exists an unknown function $\psi:\mathbb{R}\mapsto\mathbb{R}$ such that $\psi(X_i)\sim N(0,1)$ and $\psi(Y_j)\sim N(0,\sigma^2)$ for all $i=1,\dots,m$ and $j=1,\dots,n$. I'd like to estimate $\sigma^2$ in this problem.

I have obtained the following facts:

Note that $F(x)=P(X\le x)=P(\psi(X)\le \psi(x))=\Phi(\psi(x))$ where $\Phi$ is the cumulative standard normal distribution. This implies $\psi(x)=\Phi^{-1}(F(x))$.

Recall that $\psi(Y_j)\sim N(0,\sigma^2)$. So, $$\check\sigma^2=\frac1n\sum_{j=1}^n\psi^2(Y_j)$$ is an optimal estimator for $\sigma^2$. Since $F$ is unknown then I replace it with its empirical distribution function $\hat F_m$ based on $X_1,\dots,X_m$. Hence, it is natural to replace $\psi$ with $\hat\psi=\Phi^{-1}(\hat F_m)$. Therefore, I conjecture that $$\hat\sigma^2=\frac1n\sum_{j=1}^n\hat\psi^2(Y_j)$$ is an optimal estimator. I have tried to show $\check\sigma^2$ and $\hat\sigma^2$ are asymptotically equivalent, but I was failed. Could anyone help me? Or does any one have another approach?


2 Answers 2


I think the following works:

$$\hat{\sigma}^2 = \frac{1}{n}\sum_{j=1}^n \hat{\psi}^2(Y_j) = \frac{1}{n} \sum_{j=1}^n \psi^2(Y_j) + \frac{1}{n} \sum_{j=1}^n \left[\hat{\psi}^2(Y_j) - \psi^2(Y_j)\right] \qquad (1)$$

The first term is $\breve{\sigma}^2$. Then, we just need to show that the second term is asymptotically 0 (as $m \rightarrow \infty$).

Let $g(\cdot) = \Phi^{-1}(\cdot)$. We need two things. First, the Glivenko-Cantelli Theorem says that empirical cdf $\hat{F}$ converges uniformly to $F$ i.e. $\sup_t |\hat{F}(t) - F(t)| \rightarrow 0$. Second, we use the delta method. Taylor expansion of $g(\hat{F}(x))$ at $F(x)$:

$$g(\hat{F}(x)) = g(F(x)) + g'(F(x))[\hat{F}(x) - F(x)] + o_p(1)$$

The $\hat{F}(x) - F(x)$ term goes to 0 be GC theorem for all $x \implies \hat{\psi^2}(x) \rightarrow \psi^2(x)$ as $m \rightarrow \infty \implies$ the second term in (1) is 0.

  • $\begingroup$ First, I'd like to thank for your answer @bmciv. However, to prove that $\hat\sigma^2$ is optimal we have to show that $\sqrt n(\hat\sigma^2-\sigma^2)\to N(0,2\sigma^4)$, and it suffices to show $\sqrt n(\hat\sigma^2-\check\sigma^2)\to 0$ in probability because we will have $\hat\sigma^2$ and $\check\sigma^2$ have the same limiting distribution and we already had $\sqrt n(\check\sigma^2-\sigma^2)\to N(0,2\sigma^4)$. In your answer, we are only able to derive $(\hat\sigma^2-\check\sigma^2)\to 0$ in probability. So, how will we confirm the optimality of $\hat\sigma^2$? $\endgroup$
    – Jlamprong
    Jun 3, 2014 at 18:01
  • $\begingroup$ The Remainder of the Taylor Expansion is $o_p(1)$ when $\hat{F}(x) \rightarrow F(x)$. So we have to first write it in general as, say, $R_2(\hat{F}(x)-F(x))$, and then, invoking the GC theorem, to say that it too, goes to zero. And I think that by $m$ you meant $n$? $\endgroup$ Jun 5, 2014 at 11:37
  • $\begingroup$ In case $m=n$, how would it be? $\endgroup$
    – Jlamprong
    Jun 6, 2014 at 8:34

Let's try something, I think you need one more assumption to get optimality:

To have asymptotic equivalence you would need:


Using the CLT and the Delta-Method you know that:

$\sqrt{m}(\hat{\psi}^2(Y_j)-\psi^2(Y_j))=O_p(1), \forall j$ or $\hat{\psi}^2(Y_j)-\psi^2(Y_j) = O_p(1/\sqrt{m})$

Let's use some intuition, for the two estimators to be equivalent, you need the estimation error from $\hat{\psi}$ to be negligible for $n \to \infty$. In the extreme case, if $m=\infty$ then the approximation error is zero and the two are equivalent. In the other extreme case, if $m$ does not go to infinity then multiplied by $\sqrt{n}$ the approximation error from using $\hat{\psi}$ becomes infinitely large.

Assume $n^2/m \to 0$ as $m,n \to \infty$ this implies that $O_p(1/\sqrt{m})=O_p(1/n)=o_p(1/\sqrt{n})$

Then you should have:

$1/\sqrt{n}\sum_{i=1}^n[\hat{\psi}^2(Y_j)-\psi^2(Y_j)]= 1/\sqrt{n}\sum_{i=1}^n[o_p(1/\sqrt{n})] =o_p(1)$

  • $\begingroup$ Thanks @user155437 for your help. But, I'd like to ask you: 1. How did you derive $\hat{\psi}^2(Y_j)-\psi^2(Y_j) = O_p(1/\sqrt{m})$? Since using Delta-Method, I was only able so show $\hat{\psi}^2(X_j)-\psi^2(X_j) = O_p(1/\sqrt{m})$. 2. Once, (1) has been proved, how will we make sure that the sum is also $O_p(1/\sqrt{m})$? $\endgroup$
    – Jlamprong
    Jun 9, 2014 at 20:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.