3
$\begingroup$

My statistics text has the following theorem, and alludes to "appropriate conditions on the model", but never specifies what those conditions are. What conditions are necessary?

Let $\hat{\theta}_n$ denote the method of moments estimator. Under appropriate conditions on the model, the following statements hold:

  1. The estimate $\hat{\theta}_n$ exists with probability tending to $1$.

  2. The estimate is consistent: $\hat{\theta}_n \overset{P}{\to} \theta$

  3. The estimate is asymptotically normal: $\sqrt{n}(\hat{\theta}_n - \theta) \rightsquigarrow N(0,\Sigma)$

$\endgroup$
4
$\begingroup$

Almost all arguments to asymptotic normality of a sequence of statistics hinge on arguments using Taylor series, and thus, the "appropriate conditions" are generally smoothness conditions required to write the Taylor series, eliminate some of the terms, and then infer vanishing of the higher-order terms. Thus, the "appropriate conditions" usually consist of the existence of derivatives (to a desired order) for the function of interest, and the existence of higher-order moments.

Below I will give a heuristic explanation of the derivation of the asymptotic distribution for the method-of-moments estimator. Formal proof would require some greater technicality in presenting the various limiting distributions and establishing vanishing of the higher-order terms, but this heuristic demonstration ought to give you an idea of the required conditions for the result, and why these are required.


Method of moments estimation: Suppose we observe IID data from some distribution $\{ p_\theta | \theta \in \Theta \}$ with parameter $\theta = (\theta_1,...,\theta_m)$, and suppose that the first $m$ moments of this distribution exist, and can be written as functions of the parameters:

$$\mu_k \equiv \int \limits_\mathscr{X} x^k dP_\theta(x) = g_k(\theta) \quad \quad \quad \quad \quad \text{for all } k = 1,...,m.$$

For $n$ sample values, the method of moments estimator $\hat{\theta}: \mathscr{X}^n \rightarrow \Theta$ is defined by equating the first $m$ parametric moments and sample moments:

$$\hat{\mu}_k \equiv\frac{1}{n} \sum_{i=1}^n x_i^k = g_k(\hat{\theta}) \quad \quad \quad \quad \quad \quad \text{for all } k = 1,...,m.$$

Whether or not this estimator exists, and is unique, depends on how many solutions there are to this set of equations. The simplest case is where the function $g = (g_1,...,g_m)$ is invertible, with inverse denoted as $h = (h_1,...,h_m)$. This gives a unique estimator and a corresponding expression for the true parameter:

$$\theta = h (\boldsymbol{\mu}) = h (\mu_1, ..., \mu_m) \quad \quad \quad \quad \quad \hat{\theta} = h (\hat{\boldsymbol{\mu}}) = h (\hat{\mu}_1, ..., \hat{\mu}_m).$$


Asymptotic properties: For simplicity, we will assume that the function $g$ in the above setup is invertible, so we have a unique method of moments estimator. Since we are now looking at asymptotic properties we will use a subscript $n$ on the estimator. Since the data is IID, the strong law of large numbers means that the sample moments converge to the true moments almost surely:

$$\quad \quad \quad \quad \quad \quad \mathbb{E}(|X|^{2m}) < \infty \quad \quad \quad \implies \quad \quad \quad \mathbb{P} \Big( \lim_{n \rightarrow \infty} \hat{\mu}_n = \mu \Big) = 1. \text{ }$$

Thus, if $h$ is continuous and the relevant moments exist then the estimator will be strongly consistent:

$$\quad h \text{ is continuous and } \mathbb{E}(|X|^{2m}) < \infty \quad \quad \quad \implies \quad \quad \quad \mathbb{P} \Big( \lim_{n \rightarrow \infty} \hat{\theta}_n = \theta \Big) = 1. \quad \quad \quad \quad$$

If $h$ is differentiable then we can expand the function $\hat{\theta} = h (\hat{\boldsymbol{\mu}})$ using a Taylor expansion around the true moment vector $\boldsymbol{\mu}$, yielding:

$$\begin{equation} \begin{aligned} \hat{\theta}_n - \theta &= \nabla h (\boldsymbol{\mu}) (\hat{\boldsymbol{\mu}}_n - \boldsymbol{\mu}) + \mathcal{O}((\hat{\boldsymbol{\mu}}_n - \boldsymbol{\mu})^2). \\[6pt] \end{aligned} \end{equation}$$

As $n \rightarrow \infty$, the higher-order terms vanish relative to the first term. For the first term, we can invoke the central limit theorem to obtain the asymptotic distribution:

$$\quad \hat{\boldsymbol{\mu}}_n \sim \text{N} \bigg( \boldsymbol{\mu}, \frac{1}{n} \boldsymbol{\Sigma}_\boldsymbol{\mu} \bigg). \text{ }$$

Thus, by the delta method, we have the asymptotic distribution:

$$\begin{equation} \begin{aligned} \sqrt{n} (\hat{\theta}_n - \theta) &\rightarrow \sqrt{n} \cdot \nabla h (\boldsymbol{\mu}) (\hat{\boldsymbol{\mu}}_n - \boldsymbol{\mu}) \sim \text{N} \bigg( \mathbf{0}, (\nabla h (\boldsymbol{\mu}))^\text{T} \boldsymbol{\Sigma}_\boldsymbol{\mu} (\nabla h (\boldsymbol{\mu})) \bigg). \quad \quad \\[6pt] \end{aligned} \end{equation}$$

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.