3
$\begingroup$

Generalized method of moments (GMM) estimation seems to be called generalized method of moments because the standard method of moments (MoM) is a special case, following the following logic.

  1. MoM is solved by setting moment conditions equal to zero.

  2. This can be seen as a minimization problem, where the $\ell_2$ norm of the vector of moment conditions is minimized.

  3. In GMM, we might have more moment conditions than parameters, so just solving the system of equations from MoM is inadequate.

  4. However, because MoM has an equivalent formulation as a minimization problem, we can use this idea and find the moment conditions closest to zero, according to some notion of "close".

I follow the first, third, and fourth steps. However, it is not clear why the MoM calculation can be rephrased as a minimization. Why is that the case?

The second of these steps comes from this video, around the 3:30 mark.

Improve this question
$\endgroup$
4
  • 3
    $\begingroup$ This is not unique to MoM, nor even to statistics. It's a very general thing in mathematics that many problems of solving a system of equations for a like number of unknowns can also be considered as an optimization problem. For one example, consider how you solve a lot of optimization problems with calculus, by setting the vector of derivatives simultaneously to 0 and solving a system of equations (albeit there's more to it than just that). Or consider placing a convex penalty term on the set of equalities in such a way that the optimum of the new problem is at the equality you're looking for $\endgroup$
    – Glen_b
    Commented Jan 30 at 21:32
  • 1
    $\begingroup$ Very generally (and extremely trivially), any set of equations between real numbers, say $P_i(x)=Q_i(x),$ $i=1,2,\ldots,k,$ can be framed as minimizing the sum of $(P_i(x)-Q_i(x))^2$ (and showing the minimum value is $0$). I recall Richard Feynman did this somewhere in his Lectures on Physics where, by letting these equations consist of all physical laws, showed that all of physics can be expressed in terms of minimizing a single quantity! $\endgroup$
    – whuber ♦
    Commented Jan 30 at 21:40
  • $\begingroup$ In the case of GMM, I believe the idea is fairly similar - it is to construct a norm involving the quantity you want to solve for such that the minimum of the norm is the solution to the original problem. $\endgroup$
    – Glen_b
    Commented Jan 30 at 21:44
  • 1
    $\begingroup$ @whuber I believe this is related to the principle of least action. $\endgroup$
    – mhdadk
    Commented Jan 30 at 21:54

2 Answers 2

Reset to default
4
$\begingroup$

To expand on @Glen_b’s comment, consider the function $f : \mathbb R \to \mathbb R$, and suppose that we want to find the set of all $x \in \mathbb R$ such that $f(x) = 0$.

It may be the case that there are zero, one, two, or even infinitely many $x$’s that satisfy this equation. More precisely, consider the set $$ S = \{x \in \mathbb R \mid f(x) = 0\} $$ The size of $S$, denoted as $|S|$, can be $0$, finite, or infinite (countably or otherwise).

Consider the case when the set $S$ has a minimum, such that there exists an $a \in S$ and $s \geq a$ for every $s \in S$. We may then be interested in determining the value of $a$. This can be posed as the following optimization problem: $$ \begin{aligned} \min_{x \in \mathbb R} \quad & x \\ \textrm{s.t.} \quad & f(x) = 0 \end{aligned} $$ which is equivalent to $\min\{x \in \mathbb R \mid f(x) = 0\} = \min S$.

Improve this answer
$\endgroup$
2
$\begingroup$

Let $\vec m\big(\hat\theta\big\vert D)$ be the vector of $k$ moment conditions, where $D$ denotes the data and $\hat\theta$ the parameters. What we want is to find $\hat\theta$ such that $\vec m\big(\hat\theta\big\vert D)=\vec 0$. That is:

$$ \begin{bmatrix} m_1\big(\hat\theta\big\vert D)\\\vdots\\m_k\big(\hat\theta\big\vert D) \end{bmatrix} = \begin{bmatrix} 0\\\vdots\\0 \end{bmatrix} $$

Let $f\big(\hat\theta\big\vert D) = \vert\vert\vec m\big(\hat\theta\big\vert D)\vert\vert_2^2$. Then $f = m_1^2 + \dots + m_k^2$, and $\nabla f = \left(2m_1,\dots,2m_k\right)$.

To solve the minimization, we set $\nabla f = \vec 0$. That is:

$$ \begin{bmatrix} m_1\big(\hat\theta\big\vert D)\\\vdots\\m_k\big(\hat\theta\big\vert D) \end{bmatrix} = \begin{bmatrix} 0\\\vdots\\0 \end{bmatrix} $$

$\square$

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.