Two stage GMM estimator in Matlab

Question

I am trying to create a simple GMM estimator for the mean of a normally distributed random variable using the first three odd central moments of a normal distribution (all of which should be zero theoretically).

In two stage GMM, normally the first step is to minimise a least squares cost function of the errors of each individual moment condition within the sample; arriving at an initial estimate of the mean mu. The vector of errors after this first stage (evaluated at the first parameter estimate) is then used to create a covariance matrix of the moment conditions from which a weighting matrix is derived; then used in the second stage to arrive at an efficient estimate of the mean.

I have coded this up in Matlab, and to the best of my knowledge, this has been done correctly. The issue I am having is that the covariance matrix is very close to singular - meaning that it doesn't have an inverse. The 1st stage appears to work ok - the issue is with the second stage not working as required due to this issue.

Any help with diagnosing why I am running into this issue would be most appreciated!

Here is the Matlab code.

clear; close all; clc;

%% Generate normal data mu = 10; var = 4; N = 10;

X = mu + sqrt(var)*randn(N,1);

%% Function definitions for basic GMM f_cost_c = @(mu_hat,degree) (1/N)*sum((X-mu_hat).^degree);

% Specify the number of odd moments to include in cost fn N_moments = 3;

f_cost_v = @(mu_hat)f_cost_c(mu_hat,1); for i = 1:N_moments-1 f_cost_v = @(mu_hat)[f_cost_v(mu_hat); f_cost_c(mu_hat,2*i+1)]; end

f_L_cost_c = @(mu_hat) f_cost_v(mu_hat)'*f_cost_v(mu_hat);

%% Find the value of mu_hat which minimises the basic squared cost (the 1st stage of 2S GMM) mu_hat_0 = 5; mu_hat = fminunc(f_L_cost_c,mu_hat_0); mu_hat_1 = mu_hat;

%% Use the value of the cost vector at the value found in 1st stage to generate weighting matrix v_cost = f_cost_v(mu_hat);

S_hat = (1/N_moments)*(v_cost*v_cost'); W_hat = inv(S_hat);

%% 2nd stage of 2SLS GMM

% Define new cost using estimated weight matrix f_L2_cost_c = @(mu_hat) f_cost_v(mu_hat)'*W_hat*f_cost_v(mu_hat);

mu_hat_0 = mu_hat; mu_hat = fminunc(f_L2_cost_c,mu_hat_0);

Answer 1

When you calculate the weight matrix, you need to use the value of the moment conditions over the observations. Since the optimal weight matrix is the variance-co-variance matrix of your moments, you need to find that. Right now, you're just taking the inner product of the average value of your moments. Try the following:

% Note that this g function does not take the average, there is no sum,
% it is the value of the moment condition for each individual.
g_cost_c = @(mu_hat,degree) (1/N)*(X-mu_hat).^degree;
g_cost_v = @(mu_hat)g_cost_c(mu_hat,1);

for i = 1:N_moments-1
g_cost_v = @(mu_hat)[g_cost_v(mu_hat) g_cost_c(mu_hat,2*i+1)];
end

S_hat = (1/N_moments)*(v_cost*v_cost');
W_hat = inv(S_hat);

Again the difference is in where you put the expectation. The optimal weight matrix is:

$$W = E\left[ g(X_i,\theta)g(X_i,\theta)'\right]^{-1}$$

As opposed to: $$\left(E[ g(X_i,\theta)]'E[g(X_i,\theta)]\right)^{-1}$$ which is what the current code gives you.