2
$\begingroup$

Is it true that $\sqrt n (\hat{\theta}-\theta) \ \rightarrow_d \ N(0,\sigma^2)$ implies $\text{plim} \ \hat{\theta} = \theta $? If so, how can I prove this?

Attempted proof: My proof is like this:

Since $\hat{\theta} \rightarrow_{asy.} N(\theta, \frac{\sigma^2}{n})$ I have $\lim_{n \to \infty} E(\hat{\theta})= \theta \ $ and $\lim_{n \to \infty} Var(\hat{\theta})=0 $. Therefore, from convergence in quadratic mean, $\text{plim} \ \hat{\theta} = \theta $. Is this proof correct?

Improve this question
$\endgroup$
5
  • $\begingroup$ Here's something to contemplate. Let $X$ be a Rademacher$(1/2)$ variable; that is, $\Pr(X=-1)=\Pr(X=1)=1/2.$ Define the sequence $X_n = (-1)^nX, n=1,2,3,\ldots.$ All the $X_n$ are identically distributed, so they converge in distribution trivially. What random variable would they converge to in probability? Do they converge to it? Now, although this is not the situation you have, it might provide some insight into what could go wrong with your proof. $\endgroup$
    – whuber ♦
    Commented Oct 25, 2018 at 14:49
  • $\begingroup$ See stats.stackexchange.com/questions/207264/… $\endgroup$
    – Christoph Hanck
    Commented Oct 25, 2018 at 14:55
  • $\begingroup$ @whuber Thank you for your reply. $ X_n $ do not converge in probability, because for all n $ X_n $ takes -1 or 1 with probability 1/2, do they? But I don't understand how do your example relate to my proof. I would appreciate if you give me a little more information. $\endgroup$
    – David Khan
    Commented Oct 26, 2018 at 1:21
  • $\begingroup$ @christoph Thank you for your reply. I understand the proof of the link. $\endgroup$
    – David Khan
    Commented Oct 26, 2018 at 1:22
  • $\begingroup$ Let $Z_n$ be any sequence of variables that converge to a standard Normal variable. Consider the sequence $X_nZ_n.$ $\endgroup$
    – whuber ♦
    Commented Oct 26, 2018 at 13:33

1 Answer 1

Reset to default
3
$\begingroup$

We are given that

$$\sqrt n (\hat{\theta}-\theta) \to_d N(0,\sigma^2) \tag{1} $$

Make the additional assumptions that

1) For the finite distribution of $\sqrt n (\hat{\theta}-\theta)$, the $2+\delta,\; \delta >0$ absolute moment exists and is finite.

2) The sequences of 1st and 2nd moments of $\{\sqrt n (\hat{\theta}-\theta)\}$ converge each to a constant.

Then these constants are the corresponding moments of the limiting distribution. In particular, this means that

$$\lim \text {Var}[\sqrt{n}(\hat \theta -\theta)] = \sigma^2 \tag{2}$$

At the same time, we have

$$\lim \text {Var}[\sqrt{n}(\hat \theta -\theta)] = \lim \mathbb E[\sqrt{n}(\hat \theta -\theta)]^2 - \lim\left(\mathbb E[\sqrt{n}(\hat \theta -\theta)]\right)^2$$

$$= \lim \mathbb E[\sqrt{n}(\hat \theta -\theta)]^2 - \left(\lim\mathbb E[\sqrt{n}(\hat \theta -\theta)]\right)^2 = \lim \mathbb E[\sqrt{n}(\hat \theta -\theta)]^2 - 0 \tag{3}$$

Combining $(2)$ and $(3)$ we have

$$\lim \mathbb E[\sqrt{n}(\hat \theta -\theta)]^2 = \sigma^2 \implies \lim n\mathbb E(\hat \theta -\theta)^2 = \sigma^2 < \infty \implies \mathbb E(\hat \theta -\theta)^2 = O(1/n) $$

$$\implies \mathbb E(\hat \theta -\theta)^2 = o(1) \implies \mathbb E(\hat \theta -\theta)^2 \to 0 $$

The last result is convergence in quadratic mean - and convergence in quadratic mean of a random variable to a constant, implies that this constant is also its probability limit (see here for an exposition why).

So $\hat \theta_n \to_p \theta$.

While in general it does not hold that convergence in distribution implies convergence in probability, we see how, for a subset of cases and under some additional conditions, we can go from the limiting distribution to the probability limit.

Improve this answer
$\endgroup$
6
  • $\begingroup$ There seems to be a typo and we should have $E(\hat \theta - \theta)^2 = O(1/n)$. $\endgroup$
    – sonicboom
    Commented Dec 12, 2020 at 23:08
  • $\begingroup$ How strong are the assumptions 1) and 2) in the case where $\theta$ and $\hat \theta$ are the coefficients and estimated coefficients of multiple linear regression in the usual setting (e.g. finite error variance with the errors independent of the predictors)? I assume these assumptions are underlying Hansen's statement in equation (6.13)? $\endgroup$
    – sonicboom
    Commented Dec 12, 2020 at 23:29
  • 1
    $\begingroup$ @sonicbomm They do but you don't really need this because convergence in probability is proved on its own for the $Q^{-1}$ matrix. Proving the convergence in distribution is done to be able to improve on the rate of convergence, not to prove convergence in probability. $\endgroup$
    – Alecos Papadopoulos
    Commented Dec 13, 2020 at 0:40
  • $\begingroup$ I see how it works but it just seems a bit arbitrary because in 1) and 2) you make an assumption that involves $\sqrt{n}$ and ultimately this assumption results in the $O_p(n^{-1/2})$ convergence in probability and $O(1/n)$ in quadratic mean. You could have just as easily replaced the $\sqrt{n}$ with $n$ or $n^2$, etc., in 1) and 2) and obtained a different rate of convergence in probability and expectation. $\endgroup$
    – sonicboom
    Commented Dec 13, 2020 at 22:18
  • 1
    $\begingroup$ @sonicboom As I wrote you do not need to invoke them. Just prove convergence in probability directly. $\endgroup$
    – Alecos Papadopoulos
    Commented Dec 13, 2020 at 22:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.