21
$\begingroup$

Why in "Method of Moments", we equate sample moments to population moments for finding point estimator?

Where is the logic behind this?

$\endgroup$
  • 2
    $\begingroup$ It'd be nice if we had a physicist in our community to tackle this one. $\endgroup$ – mugen Dec 15 '14 at 15:10
  • 4
    $\begingroup$ @mugen, I see no relation to physics whatsoever. $\endgroup$ – Aksakal Dec 15 '14 at 15:52
  • 2
    $\begingroup$ @Aksakal they use moments of functions in physics too, and it's always nice when somebody makes a parallel for better interpretation. $\endgroup$ – mugen Dec 15 '14 at 16:30
  • 1
    $\begingroup$ As mentioned in this answer, the law of large numbers provides a justification (albeit asymptotic) for estimating a population moment by a sample moment, resulting in (often) simple, consistent estimators $\endgroup$ – Glen_b -Reinstate Monica Dec 16 '14 at 0:31
  • $\begingroup$ Ain't the whole idea is to represent the parameters using moments? Like if you try to estimate the parameter of Poisson distribution, by finding the mean (first moment) you can use it as an estimator for your parameter lambda. $\endgroup$ – denis631 Aug 9 '17 at 16:08
14
$\begingroup$

A sample consisting of $n$ realizations from identically and independently distributed random variables is ergodic. In a such a case, "sample moments" are consistent estimators of theoretical moments of the common distribution, if the theoretical moments exist and are finite.

This means that

$$\hat \mu_k(n) = \mu_k(\theta) + e_k(n), \;\;\; e_k(n) \xrightarrow{p} 0 \tag{1}$$

So by equating the theoretical moment with the corresponding sample moment we have

$$\hat \mu_k(n) = \mu_k(\theta) \Rightarrow \hat \theta(n) = \mu_k^{-1}(\hat \mu_k(n)) = \mu_k^{-1}[\mu_k(\theta) + e_k(n)]$$

So ($\mu_k$ does not depend on $n$)

$$\text{plim} \hat \theta(n) = \text{plim}\big[\mu_k^{-1}(\mu_k(\theta) + e_k)\big] = \mu_k^{-1}\big(\mu_k(\theta) + \text{plim}e_k(n)\big)$$

$$=\mu_k^{-1}\big(\mu_k(\theta) + 0\big) = \mu_k^{-1}\mu_k(\theta) = \theta$$

So we do that because we obtain consistent estimators for the unknown parameters.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ what does "plim" mean ? I am not familiar with "p" in $e_k(n) \xrightarrow{p} 0$ $\endgroup$ – user 31466 Dec 16 '14 at 8:59
  • $\begingroup$ @leaf probability limit $\endgroup$ – Alecos Papadopoulos Dec 16 '14 at 11:00
  • $\begingroup$ What would be happen if it was regular limit instead of probability limit? $\endgroup$ – user 31466 Dec 16 '14 at 12:05
  • $\begingroup$ It would tell us that the estimator becomes a constant, not that it tends probabilistically to one. Perhaps you should look up modes of convergence of random variables, wikipedia has a decent introduction, en.wikipedia.org/wiki/Convergence_of_random_variables $\endgroup$ – Alecos Papadopoulos Dec 16 '14 at 13:28
  • 1
    $\begingroup$ @AlecosPapadopoulos Agreed. I'm wondering then whether it makes sense to put something simple like "... and under certain conditions on $\mu_k$"? $\endgroup$ – Jerome Baum Dec 20 '14 at 4:26
12
$\begingroup$

Econometricians call this "the analogy principle". You compute the population mean as the expected value with respect to the population distribution; you compute the estimator as the expected value with respect to the sample distribution, and it turns out to be the sample mean. You have a unified expression $$ T(F) = \int t(x) \, {\rm d}F(x) $$ into which you plug either the population $F(x)$, say $F(x) = \int_{\infty}^x \frac1{\sqrt{2\pi\sigma^2}} \exp\bigl[ - \frac{(u-\mu)^2}{2\sigma^2} \bigr] \, {\rm d}u $ or the sample $F_n(x) = \frac 1n \sum_{i=1}^n 1\{ x_i \le x \}$, so that ${\rm d}F_n(x)$ is a bunch of delta-functions, and the (Lebesgue) integral with respect to ${\rm d}F_n(x)$ is the sample sum $\frac1n \sum_{i=1}^n t(x_i)$. If your functional $T(\cdot)$ is (weakly) differentiable, and $F_n(x)$ converges in the appropriate sense to $F(x)$, then it is easy to establish that the estimate is consistent, although of course more hoopla is needed to obtain say asymptotic normality.

| cite | improve this answer | |
$\endgroup$
  • 1
    $\begingroup$ I haven't heard this called "analogy principle", but it is an often used econometric analysis pattern indeed: plug the sample estimator whenever the population parameter is needed but unknown. $\endgroup$ – Aksakal Dec 15 '14 at 15:56
  • $\begingroup$ @Aksakal:"plug the sample estimator whenever the population parameter is needed but unknown." isn't this approach simply called statistics? $\endgroup$ – user603 Dec 16 '14 at 0:36
  • $\begingroup$ @user603: No, not. There are other alternative approaches, and plu-in estimators can be bad. $\endgroup$ – kjetil b halvorsen Dec 25 '16 at 17:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.