4
$\begingroup$

The idea behind Method of Moments (MOM) is quite intuitive:

find the parameter values so that the population moments (which are functions of those parameters of interest) matches the sample moments.

But why moments? My hunch is that moments are used because they capture the essence of the DGP. (More formalization of this hunch would be appreciated!) This begs the question: is there "something else" that can capture the essence of the DGP? In other words, can we

find the parameter values so that the population "something else" (which are functions of those parameters of interest) matches the sample "something else"?

Is it correct to think that the likelihood is one such "something else", thus motivating MLE? Or could quantile be one such "something else", thus motivating a "method of quantiles"?

Improve this question
$\endgroup$
4
  • 2
    $\begingroup$ A likelihood is not a population property. It's unclear what you could mean by "essence of the DGP," given that a moment of any monotonic transformation of the data values also works in MoM. Perhaps what makes working with moments attractive and generalizable is that they are all averages. $\endgroup$
    – whuber ♦
    Commented Jun 28, 2021 at 22:13
  • $\begingroup$ Method of Moments are particularly attractive to those who like unbiased estimators. But it can in some particular cases lead to absurd results. $\endgroup$
    – Henry
    Commented Jun 28, 2021 at 22:48
  • $\begingroup$ @whuber By "essence of the DGP", I mean the something like the "ways to sufficiently characterize the distribution of the data". Moments can be one such way (understanding that moments don't completely characterize the distribution). Is there any other way? For example, can we have a "method of quantiles"? $\endgroup$
    – Heisenberg
    Commented Jun 29, 2021 at 7:01
  • 1
    $\begingroup$ One way to view moments is that they are coefficients of a Hilbert basis. Choose another basis and you get a different set of coefficients. Fourier analysis can be understood in this way, for instance. But guessing what might work is not a fruitful avenue of investigation. Instead, study the conceptual and theoretical bases of statistical inference and decision theory to find ways to develop and evaluate methods of inference. $\endgroup$
    – whuber ♦
    Commented Jun 30, 2021 at 13:47

3 Answers 3

Reset to default
2
$\begingroup$

Yes, you can have a 'method of quantiles'. In a $p$-dimensional parametric family you can set $p$ observed quantiles equal to their expected values and (for most choices of the quantiles) get consistent estimators. For example, you can estimate the parameter in any location family using the median instead of the mean. Sometimes the mean will be more efficient (eg, Normal), sometimes the median will be more efficient (eg, Laplace). Similarly, you can estimate a scale parameter using differences in quantiles (eg, interquartile range).

There are technical annoyances about quantiles that make them less popular. If you define a quantile by a minimisation (minimise sum of absolute values of residuals for the median) there isn't always a unique solution. Alternatively, if you define it by solving an equation (sum of +1 for positive residuals and -1 for negative residuals) there isn't always an exact solution to the equation. Also, the functions involved aren't smooth. These aren't fatal objections, but they do make the maths harder.

More generally, maximum likelihood estimators and other M-estimators use means of summaries that (in general) aren't just powers of the data. Part of the attractiveness of the method of moments was probably that it gave maximum likelihood estimators for some interesting models (exponential family models) but was computationally tractable in some cases where the maximum likelihood estimator wasn't. In one-dimensional problems there are also L-estimators (linear combinations of quantiles) and R-estimators (minimisation of rank test statistics)

Improve this answer
$\endgroup$
1
$\begingroup$

Note that two RVs can have equal $k$-th moments for all $k=1,2,...$ and still be different, so they don't always capture "the essence" of the distribution. There are many counterexamples online but also in Casella and Berger.

However, ff the moment generating function is defined in an interval around $0$, then equality of moments provides equality of distributions.

Improve this answer
$\endgroup$
1
  • $\begingroup$ My intuition is that, while moments don't completely characterize the distribution, they do so sufficiently well for methods of moments to make sense. My question is then, other than moments, is there anything else that "sufficiently characterize the distribution of the data"? For example, is there such a thing as "method of quantiles"? $\endgroup$
    – Heisenberg
    Commented Jun 29, 2021 at 7:02
1
$\begingroup$

The idea of the "method of moments" is indeed not restricted to moments, but can be applied to any estimator for a summary statistic. For instance, Elo derived an estimator for chess ratings by approximately solving the relationship between ratings (= model parameters) and mean wins in a Thurstonian paired comparison model (unfortunately, in his book, Elo did not make the relationship clear to previous work by others).

Which summary statistics to use typically depends on what is observable. But even in the original form of the method of moments, there is some arbitrariness: you always have only few parameters, but an infinite set of moments, so there are an infinite number of ways to compute parameter estimators with the method of moments.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.