There are several methods to estimate the parameters of a model. This is a core part of statistics/econometrics. GMM (Generalized Method of Moments) is one such method and it is more robust (statistically and literally[for non-statistics audience]) than several others.
It should be intuitive that the process of estimation involves how good your model fits the data. The GMM uses more conditions than the ordinary models while doing this.
(You have mentioned average and variance. I am assuming that is a familiar idea). Average and Variance are some basic metrics of the data. A person models the data to understand it's nature. A perfect(hypothetical model) would explain the data through and through.
Let us take an example of modeling heights of all the people in a building. There are two metrics average and variance. Average is the first level metric, variance is the second level metric. An average is adding all the heights and dividing it by the number of people. It tells you something like 11 feet is ridiculous. 5 feet is sensible.
Now consider the variance, it will tell an additional layer of information: 6 feet is not ridiculous(based on average) but how likely is it for the height of the person to be 6 feet. If the building is a middle school building, it is less likely right ? If it is office building more likely.
These are examples of something technically called moments of the data(after explaining average and variance, should be comfortable ?). One's model should do well if it caters to these conditions of average and variance observed. Beyond average and variance, there are several other metrics.
The GMM fits the model for these higher metrics(moments). Simpler methods cater to smaller metrics. The name as it suggests is generalized method - it tries to be as general as possible.