I am trying to create a simple GMM estimator for the mean of a normally distributed random variable using the first three odd central moments of a normal distribution (all of which should be zero theoretically).

In two stage GMM, normally the first step is to minimise a least squares cost function of the errors of each individual moment condition within the sample; arriving at an initial estimate of the mean mu. The vector of errors after this first stage (evaluated at the first parameter estimate) is then used to create a covariance matrix of the moment conditions from which a weighting matrix is derived; then used in the second stage to arrive at an efficient estimate of the mean.

I have coded this up in Matlab, and to the best of my knowledge, this has been done correctly. The issue I am having is that the covariance matrix is very close to singular - meaning that it doesn't have an inverse. The 1st stage appears to work ok - the issue is with the second stage not working as required due to this issue.

Any help with diagnosing why I am running into this issue would be most appreciated!

Here is the Matlab code.

clear; close all; clc;

%% Generate normal data mu = 10; var = 4; N = 10;

X = mu + sqrt(var)*randn(N,1);

%% Function definitions for basic GMM f_cost_c = @(mu_hat,degree) (1/N)*sum((X-mu_hat).^degree);

% Specify the number of odd moments to include in cost fn N_moments = 3;

f_cost_v = @(mu_hat)f_cost_c(mu_hat,1); for i = 1:N_moments-1 f_cost_v = @(mu_hat)[f_cost_v(mu_hat); f_cost_c(mu_hat,2*i+1)]; end

f_L_cost_c = @(mu_hat) f_cost_v(mu_hat)'*f_cost_v(mu_hat);

%% Find the value of mu_hat which minimises the basic squared cost (the 1st stage of 2S GMM) mu_hat_0 = 5; mu_hat = fminunc(f_L_cost_c,mu_hat_0); mu_hat_1 = mu_hat;

%% Use the value of the cost vector at the value found in 1st stage to generate weighting matrix v_cost = f_cost_v(mu_hat);

S_hat = (1/N_moments)*(v_cost*v_cost'); W_hat = inv(S_hat);

%% 2nd stage of 2SLS GMM

% Define new cost using estimated weight matrix f_L2_cost_c = @(mu_hat) f_cost_v(mu_hat)'*W_hat*f_cost_v(mu_hat);

mu_hat_0 = mu_hat; mu_hat = fminunc(f_L2_cost_c,mu_hat_0);

1 Answer 1


When you calculate the weight matrix, you need to use the value of the moment conditions over the observations. Since the optimal weight matrix is the variance-co-variance matrix of your moments, you need to find that. Right now, you're just taking the inner product of the average value of your moments. Try the following:

% Note that this g function does not take the average, there is no sum,
% it is the value of the moment condition for each individual.
g_cost_c = @(mu_hat,degree) (1/N)*(X-mu_hat).^degree;
g_cost_v = @(mu_hat)g_cost_c(mu_hat,1);

for i = 1:N_moments-1
g_cost_v = @(mu_hat)[g_cost_v(mu_hat) g_cost_c(mu_hat,2*i+1)];

S_hat = (1/N_moments)*(v_cost*v_cost');
W_hat = inv(S_hat);

Again the difference is in where you put the expectation. The optimal weight matrix is:

$$W = E\left[ g(X_i,\theta)g(X_i,\theta)'\right]^{-1}$$

As opposed to: $$\left(E[ g(X_i,\theta)]'E[g(X_i,\theta)]\right)^{-1}$$ which is what the current code gives you.

  • $\begingroup$ That's great - thank you so much. A silly mistake by me! $\endgroup$
    – ben18785
    Aug 21, 2014 at 21:09
  • $\begingroup$ No problem, happens to all of us once in awhile! $\endgroup$
    – jayk
    Aug 21, 2014 at 23:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.