Why in "Method of Moments", we equate sample moments to population moments for finding point estimator?
Where is the logic behind this?
$\begingroup$
$\endgroup$
6
-
2$\begingroup$ It'd be nice if we had a physicist in our community to tackle this one. $\endgroup$– mugenDec 15, 2014 at 15:10
-
5$\begingroup$ @mugen, I see no relation to physics whatsoever. $\endgroup$– AksakalDec 15, 2014 at 15:52
-
2$\begingroup$ @Aksakal they use moments of functions in physics too, and it's always nice when somebody makes a parallel for better interpretation. $\endgroup$– mugenDec 15, 2014 at 16:30
-
2$\begingroup$ As mentioned in this answer, the law of large numbers provides a justification (albeit asymptotic) for estimating a population moment by a sample moment, resulting in (often) simple, consistent estimators $\endgroup$– Glen_bDec 16, 2014 at 0:31
-
1$\begingroup$ Ain't the whole idea is to represent the parameters using moments? Like if you try to estimate the parameter of Poisson distribution, by finding the mean (first moment) you can use it as an estimator for your parameter lambda. $\endgroup$– denis631Aug 9, 2017 at 16:08
|
Show 1 more comment
3 Answers
$\begingroup$
$\endgroup$
9
A sample consisting of $n$ realizations from identically and independently distributed random variables is ergodic. In a such a case, "sample moments" are consistent estimators of theoretical moments of the common distribution, if the theoretical moments exist and are finite.
This means that
$$\hat \mu_k(n) = \mu_k(\theta) + e_k(n), \;\;\; e_k(n) \xrightarrow{p} 0 \tag{1}$$
So by equating the theoretical moment with the corresponding sample moment we have
$$\hat \mu_k(n) = \mu_k(\theta) \Rightarrow \hat \theta(n) = \mu_k^{-1}(\hat \mu_k(n)) = \mu_k^{-1}[\mu_k(\theta) + e_k(n)]$$
So ($\mu_k$ does not depend on $n$)
$$\text{plim} \hat \theta(n) = \text{plim}\big[\mu_k^{-1}(\mu_k(\theta) + e_k)\big] = \mu_k^{-1}\big(\mu_k(\theta) + \text{plim}e_k(n)\big)$$
$$=\mu_k^{-1}\big(\mu_k(\theta) + 0\big) = \mu_k^{-1}\mu_k(\theta) = \theta$$
So we do that because we obtain consistent estimators for the unknown parameters.
answered Dec 15, 2014 at 15:40
-
$\begingroup$ what does "plim" mean ? I am not familiar with "p" in $e_k(n) \xrightarrow{p} 0$ $\endgroup$ Dec 16, 2014 at 8:59
-
-
$\begingroup$ What would be happen if it was regular limit instead of probability limit? $\endgroup$ Dec 16, 2014 at 12:05
-
1$\begingroup$ It would tell us that the estimator becomes a constant, not that it tends probabilistically to one. Perhaps you should look up modes of convergence of random variables, wikipedia has a decent introduction, en.wikipedia.org/wiki/Convergence_of_random_variables $\endgroup$ Dec 16, 2014 at 13:28
-
1$\begingroup$ @AlecosPapadopoulos Agreed. I'm wondering then whether it makes sense to put something simple like "... and under certain conditions on $\mu_k$"? $\endgroup$ Dec 20, 2014 at 4:26
$\begingroup$
$\endgroup$
3
Econometricians call this "the analogy principle". You compute the population mean as the expected value with respect to the population distribution; you compute the estimator as the expected value with respect to the sample distribution, and it turns out to be the sample mean. You have a unified expression $$ T(F) = \int t(x) \, {\rm d}F(x) $$ into which you plug either the population $F(x)$, say $F(x) = \int_{\infty}^x \frac1{\sqrt{2\pi\sigma^2}} \exp\bigl[ - \frac{(u-\mu)^2}{2\sigma^2} \bigr] \, {\rm d}u $ or the sample $F_n(x) = \frac 1n \sum_{i=1}^n 1\{ x_i \le x \}$, so that ${\rm d}F_n(x)$ is a bunch of delta-functions, and the (Lebesgue) integral with respect to ${\rm d}F_n(x)$ is the sample sum $\frac1n \sum_{i=1}^n t(x_i)$. If your functional $T(\cdot)$ is (weakly) differentiable, and $F_n(x)$ converges in the appropriate sense to $F(x)$, then it is easy to establish that the estimate is consistent, although of course more hoopla is needed to obtain say asymptotic normality.
-
1$\begingroup$ I haven't heard this called "analogy principle", but it is an often used econometric analysis pattern indeed: plug the sample estimator whenever the population parameter is needed but unknown. $\endgroup$– AksakalDec 15, 2014 at 15:56
-
$\begingroup$ @Aksakal:"plug the sample estimator whenever the population parameter is needed but unknown." isn't this approach simply called statistics? $\endgroup$– user603Dec 16, 2014 at 0:36
-
$\begingroup$ @user603: No, not. There are other alternative approaches, and plug-in estimators can be bad. $\endgroup$ Dec 25, 2016 at 17:55
$\begingroup$
$\endgroup$
1
I might be wrong but the way that I think about it is as follows:
Let's say you have samples $X_1, X_2, \dotsc, X_n$. Then, the method of the moments suggest that we should compare $m-th$ moment of sample with $m$th moment of population
$$(X_1 + X_2 + \dotsm + X_n) / n = μ$$
here, we are averaging all the samples out which seems like a good estimate of population mean
$$(X_1^2 + X_2^2 + \dotsm + X_n^2) / n = σ^2$$
here, we are averaging all the sample variances out which also seems like a good estimate of population variance
And so on,
That is what I think is a good explanation of how method of moment works!
-
$\begingroup$ In the first equation you compare first moments, not $m^\text{th}$ moments. In the last equation you aren't averaging variances and they are not sample moments, either. $\endgroup$– whuber ♦Mar 3, 2022 at 17:10