8
$\begingroup$

From my understanding, both are estimators that are based on first providing an unbiased statistic $T(X)$ and obtaining the root to the equation:

$$c(X) \left( T(X) - E(T(X)) \right) = 0$$

Secondly both are in some sense "nonparametric" in that, regardless of what the actual probability model for $X$ may be, if you think of $T(\cdot)$ as a meaningful summary of the data, then you will be consistently estimating that "thing" regardless of whether that thing has any probabilistic connection with the actual probability model for the data. (e.g. estimating the sample mean from Weibull distributed failure times without censoring).

However, method of moments seems to insinuate that the $T(X)$ of interest must be a moment for a readily assumed probability model, however, one estimates it with an estimating equation and not maximum likelihood (even though they may agree, as is the case for means of normally distributed random variables). Calling something a "moment" to me has the connotation of insinuating a probability model. However, supposing for instance we have log normally distributed data, is the method of moments estimator for the 3rd central moment based on the 3rd sample moment, e.g. $$\hat{\mu_3} = \frac{1}{n}\sum_{i=1}^n \left( X_i - \bar{X} \right)^3$$

Or does one estimate the first and second moment, transform them to estimate the probability model parameters, $\mu$ and $\sigma$ (whose estimates I will denote with hat notation) and then use these estimates as plug-ins for the derive skewness of lognormal data, i.e.

$$ \hat{\mu_3} = \left( \exp \left( \hat{\sigma}^2 \right) + 2\right) \sqrt{\exp \left( \hat{\sigma}^2-1\right)}$$

Improve this question
$\endgroup$

2 Answers 2

Reset to default
5
$\begingroup$

The most common justification of the method of moments is simply the law of large numbers, which would seem to make your suggestion of estimating $\mu_3$ by $\hat{\mu}_3$ "method of moments" (and I'd be inclined to call it MoM in any case).

However, a number of books and documents, such as this for example (and to some extent the wikipedia page on method of moments) imply that you take the lowest $k$ moments* and estimate the required quantities for given the probability model from that, as you imply by estimating $\mu_3$ from the first two moments.

*(where you need to estimate $k$ parameters to obtain the required quantity)

--

Ultimately, I guess it comes down to "who defines what counts as method of moments?"

Do we look to Pearson? Do we look to the most common conventions? Do we accept any convenient definition? --- Any of those choices has problems, and benefits.

The interesting bit, to me, is whether one can always or almost always reparameterize a parametric family to characterize an estimation problem in EE as the solution to the moments of a (possibly bizarre) distribution function?

Clearly there are large classes of distribution for which method of moments would be useless.

For an obvious example, the mean of the Cauchy distribution is undefined.

Even when moments exist and are finite, there could be a large number of situations where the set of equations $f(\mathbf{\theta},\mathbf{y})=0$ has 0 solutions (think of some curve that never crosses the x-axis) or multiple solutions (one that crosses the axis repeatedly -- though multiple solutions aren't necessarily an insurmountable problem if you have a way to choose between them).

Of course, we also commonly see situations where a solution exists but doesn't lie in the parameter space (there may even be cases where there's never a solution in the parameter space, but I don't know of any -- it would be an interesting question to discover if some such cases exist).

I imagine there can be more complicated situations still, though I don't have any in mind at the moment.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ The wikipedia article seems to say that MoM is used when you are interested in estimating the DF of iid data, $X_1, X_2, \ldots, X_n \sim_{iid} F$ where $F$ has a known parametric distribution and by estimating the necessary $k$ moments for solving systems of equations for the $p$ parameters, $p < k???$ you get the appropriate estimator. The interesting bit, to me, is whether one can always or almost always reparameterize a parametric family to characterize an estimation problem in EE as the solution to the moments of a (possibly bizarre) distribution function? $\endgroup$
    – AdamO
    Commented Nov 3, 2014 at 19:18
  • $\begingroup$ Hi AdamO -- I wrote a comment but it started to get too long so I moved it up to the end of my answer. If you can expand on where that interest in your last sentence lies, I think that might make for some stimulating areas to explore (I don't claim to know any more than you on this, but I'd love to try to learn more). $\endgroup$
    – Glen_b
    Commented Nov 3, 2014 at 21:06
1
$\begingroup$

Estimating equations is a more general method, it doesnt specify from where you get the estimating equation. Maximum likelihood is also an example of estimating equations, as it leads to the score equation. Various forms of quasi (or pseudo)-likelihood are other examples! as are method of moments.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ Likelihoods with local extrema do not have unique solutions when solving their corresponding score functions (but directly maximizing the likelihood would address this), so to me, there are some pathological cases where ML is more general than EE... but for somewhat regular parametric models, their ML solution can be obtained with score equations as EEs. I am not as familiar with MoM. $\endgroup$
    – AdamO
    Commented Nov 3, 2014 at 23:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.