24
$\begingroup$

Why in "Method of Moments", we equate sample moments to population moments for finding point estimator?

Where is the logic behind this?

$\endgroup$
6
  • 2
    $\begingroup$ It'd be nice if we had a physicist in our community to tackle this one. $\endgroup$
    – mugen
    Commented Dec 15, 2014 at 15:10
  • 5
    $\begingroup$ @mugen, I see no relation to physics whatsoever. $\endgroup$
    – Aksakal
    Commented Dec 15, 2014 at 15:52
  • 2
    $\begingroup$ @Aksakal they use moments of functions in physics too, and it's always nice when somebody makes a parallel for better interpretation. $\endgroup$
    – mugen
    Commented Dec 15, 2014 at 16:30
  • 2
    $\begingroup$ As mentioned in this answer, the law of large numbers provides a justification (albeit asymptotic) for estimating a population moment by a sample moment, resulting in (often) simple, consistent estimators $\endgroup$
    – Glen_b
    Commented Dec 16, 2014 at 0:31
  • 1
    $\begingroup$ Ain't the whole idea is to represent the parameters using moments? Like if you try to estimate the parameter of Poisson distribution, by finding the mean (first moment) you can use it as an estimator for your parameter lambda. $\endgroup$
    – denis631
    Commented Aug 9, 2017 at 16:08

3 Answers 3

16
$\begingroup$

A sample consisting of $n$ realizations from identically and independently distributed random variables is ergodic. In a such a case, "sample moments" are consistent estimators of theoretical moments of the common distribution, if the theoretical moments exist and are finite.

This means that

$$\hat \mu_k(n) = \mu_k(\theta) + e_k(n), \;\;\; e_k(n) \xrightarrow{p} 0 \tag{1}$$

So by equating the theoretical moment with the corresponding sample moment we have

$$\hat \mu_k(n) = \mu_k(\theta) \Rightarrow \hat \theta(n) = \mu_k^{-1}(\hat \mu_k(n)) = \mu_k^{-1}[\mu_k(\theta) + e_k(n)]$$

So ($\mu_k$ does not depend on $n$)

$$\text{plim} \hat \theta(n) = \text{plim}\big[\mu_k^{-1}(\mu_k(\theta) + e_k)\big] = \mu_k^{-1}\big(\mu_k(\theta) + \text{plim}e_k(n)\big)$$

$$=\mu_k^{-1}\big(\mu_k(\theta) + 0\big) = \mu_k^{-1}\mu_k(\theta) = \theta$$

So we do that because we obtain consistent estimators for the unknown parameters.

$\endgroup$
9
  • $\begingroup$ what does "plim" mean ? I am not familiar with "p" in $e_k(n) \xrightarrow{p} 0$ $\endgroup$
    – user 31466
    Commented Dec 16, 2014 at 8:59
  • $\begingroup$ @leaf probability limit $\endgroup$ Commented Dec 16, 2014 at 11:00
  • $\begingroup$ What would be happen if it was regular limit instead of probability limit? $\endgroup$
    – user 31466
    Commented Dec 16, 2014 at 12:05
  • 1
    $\begingroup$ It would tell us that the estimator becomes a constant, not that it tends probabilistically to one. Perhaps you should look up modes of convergence of random variables, wikipedia has a decent introduction, en.wikipedia.org/wiki/Convergence_of_random_variables $\endgroup$ Commented Dec 16, 2014 at 13:28
  • 1
    $\begingroup$ @AlecosPapadopoulos Agreed. I'm wondering then whether it makes sense to put something simple like "... and under certain conditions on $\mu_k$"? $\endgroup$
    – Anonymous
    Commented Dec 20, 2014 at 4:26
13
$\begingroup$

Econometricians call this "the analogy principle". You compute the population mean as the expected value with respect to the population distribution; you compute the estimator as the expected value with respect to the sample distribution, and it turns out to be the sample mean. You have a unified expression $$ T(F) = \int t(x) \, {\rm d}F(x) $$ into which you plug either the population $F(x)$, say $F(x) = \int_{\infty}^x \frac1{\sqrt{2\pi\sigma^2}} \exp\bigl[ - \frac{(u-\mu)^2}{2\sigma^2} \bigr] \, {\rm d}u $ or the sample $F_n(x) = \frac 1n \sum_{i=1}^n 1\{ x_i \le x \}$, so that ${\rm d}F_n(x)$ is a bunch of delta-functions, and the (Lebesgue) integral with respect to ${\rm d}F_n(x)$ is the sample sum $\frac1n \sum_{i=1}^n t(x_i)$. If your functional $T(\cdot)$ is (weakly) differentiable, and $F_n(x)$ converges in the appropriate sense to $F(x)$, then it is easy to establish that the estimate is consistent, although of course more hoopla is needed to obtain say asymptotic normality.

$\endgroup$
3
  • 1
    $\begingroup$ I haven't heard this called "analogy principle", but it is an often used econometric analysis pattern indeed: plug the sample estimator whenever the population parameter is needed but unknown. $\endgroup$
    – Aksakal
    Commented Dec 15, 2014 at 15:56
  • $\begingroup$ @Aksakal:"plug the sample estimator whenever the population parameter is needed but unknown." isn't this approach simply called statistics? $\endgroup$
    – user603
    Commented Dec 16, 2014 at 0:36
  • $\begingroup$ @user603: No, not. There are other alternative approaches, and plug-in estimators can be bad. $\endgroup$ Commented Dec 25, 2016 at 17:55
0
$\begingroup$

I might be wrong but the way that I think about it is as follows:

Let's say you have samples $X_1, X_2, \dotsc, X_n$. Then, the method of the moments suggest that we should compare $m-th$ moment of sample with $m$th moment of population

$$(X_1 + X_2 + \dotsm + X_n) / n = μ$$

here, we are averaging all the samples out which seems like a good estimate of population mean

$$(X_1^2 + X_2^2 + \dotsm + X_n^2) / n = σ^2$$

here, we are averaging all the sample variances out which also seems like a good estimate of population variance

And so on,

That is what I think is a good explanation of how method of moment works!

$\endgroup$
1
  • $\begingroup$ In the first equation you compare first moments, not $m^\text{th}$ moments. In the last equation you aren't averaging variances and they are not sample moments, either. $\endgroup$
    – whuber
    Commented Mar 3, 2022 at 17:10

Not the answer you're looking for? Browse other questions tagged or ask your own question.