# Gradient in GMM estimation

I have a question that might be trivial but I have not much knowledge on that method:

I want to estimate a structural model with GMM and my model works in the sense that it estimated the right coefficients of simulated data. This works fine without adding a gradient (I'm using "gmm"-package in R), just the vcov-matrix of the parameters differs.

So I'm a little bit confused which matrix is "the right" one and if I have to add the gradient or not.

• The documentation says: "By default, the numerical algorithm numericDeriv is used. It is of course strongly suggested to provide this function when it is possible. This gradient is used to compute the asymptotic covariance matrix of \hat{θ} and to obtain the analytical gradient of the objective function if the method is set to "CG" or "BFGS" in optim and if "type" is not set to "cue"" So obviously R solves this numerically if I don't provide it!? I do not recognize any difference in performance, so letting R do the job removes at least the error source of getting the gradient wrong. Comments? – Ahlerich Dec 9 '13 at 2:23
• Could you spell out the abbreviation GMM. – Sextus Empiricus Nov 6 '18 at 8:16

## 1 Answer

I'm going to start with some notation. Say the moments you are using are of the form $\operatorname{E}[g(x_t,\theta)]=0$, where $\theta$ are the parameters you're estimating. You'll have some weight matrix $W$, which will be positive-definite. The objective function you minimize to get your estimate will then be:

$$\bigg(\frac{1}{T}\sum_{t=1}^T g(x_t,\theta)\bigg)' W \bigg(\frac{1}{T}\sum_{t=1}^T g(x_t,\theta)\bigg)$$

Once you have some estimate $\hat\theta$, the asymptotic variance of that estimate will be:

$$(G'WG)^{-1}G'W\Omega WG(G'WG)^{-1}$$

where $G = \operatorname{E}[\nabla_{\theta}g(Y_t,\theta_0)]$ (the gradient of your moment conditions) and $\Omega = \operatorname{E}[g(x_t,\theta_0)g(x_t,\theta_0)']$ (the variance-covariance matrix of your moment conditions).

## How parameter variance-covariance matrices can differ between estimates

There are two major reasons your variance covariance matrix could differ between estimations, even if both estimations are "right":

1. Calculation of the gradient can be different. This seems to be most relevant to your question. You need to estimate $G$ somehow in order to calculate that variance-covariance matrix. For complex versions of $g(x_t,\theta)$ there may not be a simple function available for calculating the gradient. In this case you have to use a numerical approximation (this gmm package uses the one included in numDeriv). The approximation of the gradient will always have some error, so if you have a closed form for the gradient use that instead. Different methods for approximating the gradient matrix could lead to different results as well. Numerical derivatives are a tricky business.
2. Choice of $W$. Choice of a weight matrix, $W$, can effect your estimate. The optimal $W$ is always $\Omega^{-1}$. Many choose to start with the identity matrix. I'm assuming this isn't your problem.
3. Different methods of updating $W$. If you're lucky then $\Omega^{-1}$ does not depend on $\theta$, but in general this isn't the case. There are different options on how to deal with $W$, many involve choosing an initial $W$ (often the identity matrix) and updating some number of times with $W=\hat\Omega^{-1}$. Common choices are "two-step", "iterative" and "continuously-updating" GMM. You can see a classic comparison of the three in "Finite-sample properties of some alternative GMM estimators" by Hansen, Heaton and Yaron. It looks like the gmm package defaults to two-step, meaning it starts with the $W$ you gives it, converges to an estimate, updates $W$ with $\hat\Omega^{-1}$ and converges to a new estimate. If your procedure for updating $W$ differs you're likely to get different results, just as if you start with a different $W$.

Any of these problems could produce different estimates for the variance-covariance matrix but very close estimates for the parameters. My advice is to use the analytic gradient if you have it. Choice of the updating often depends on the problem. Try continuously updating weight matrix if possible (sometimes this method can take awhile). If you can't use continuously updating, also try the iterative procedure, which is just like doing the two-step over and over again until your weight matrix converges over the updates.