What's special about moments that allows "method of moments" to work? The idea behind Method of Moments (MOM) is quite intuitive:
find the parameter values so that the population moments (which are functions of those parameters of interest) matches the sample moments.
But why moments? My hunch is that moments are used because they capture the essence of the DGP. (More formalization of this hunch would be appreciated!) This begs the question: is there "something else" that can capture the essence of the DGP? In other words, can we
find the parameter values so that the population "something else" (which are functions of those parameters of interest) matches the sample "something else"?
Is it correct to think that the likelihood is one such "something else", thus motivating MLE? Or could quantile be one such "something else", thus motivating a "method of quantiles"?
 A: Note that two RVs can have equal $k$-th moments for all $k=1,2,...$ and still be different, so they don't always capture "the essence" of the distribution. There are many counterexamples online but also in Casella and Berger.
However, ff the moment generating function is defined in an interval around $0$, then equality of moments provides equality of distributions.
A: Yes, you can have a 'method of quantiles'.  In a $p$-dimensional parametric family you can set $p$ observed quantiles equal to their expected values and (for most choices of the quantiles) get consistent estimators. For example, you can estimate the parameter in any location family using the median instead of the mean. Sometimes the mean will be more efficient (eg, Normal), sometimes the median will be more efficient (eg, Laplace). Similarly, you can estimate a scale parameter using differences in quantiles (eg, interquartile range).
There are technical annoyances about quantiles that make them less popular. If you define a quantile by a minimisation (minimise sum of absolute values of residuals for the median) there isn't always a unique solution. Alternatively, if you define it by solving an equation (sum of +1 for positive residuals and -1 for negative residuals) there isn't always an exact solution to the equation.  Also, the functions involved aren't smooth. These aren't fatal objections, but they do make the maths harder.
More generally, maximum likelihood estimators and other M-estimators use means of summaries that (in general) aren't just powers of the data. Part of the attractiveness of the method of moments was probably that it gave maximum likelihood estimators for some interesting models (exponential family models) but was computationally tractable in some cases where the maximum likelihood estimator wasn't. In one-dimensional problems there are also L-estimators (linear combinations of quantiles) and R-estimators (minimisation of rank test statistics)
A: The idea of the "method of moments" is indeed not restricted to moments, but can be applied to any estimator for a summary statistic. For instance, Elo derived an estimator for chess ratings by approximately solving the relationship between ratings (= model parameters) and mean wins in a Thurstonian paired comparison model (unfortunately, in his book, Elo did not make the relationship clear to previous work by others).
Which summary statistics to use typically depends on what is observable. But even in the original form of the method of moments, there is some arbitrariness: you always have only few parameters, but an infinite set of moments, so there are an infinite number of ways to compute parameter estimators with the method of moments.
