I am trying to create a simple GMM estimator for the mean of a normally distributed random variable using the first three odd central moments of a normal distribution (all of which should be zero theoretically).
In two stage GMM, normally the first step is to minimise a least squares cost function of the errors of each individual moment condition within the sample; arriving at an initial estimate of the mean mu. The vector of errors after this first stage (evaluated at the first parameter estimate) is then used to create a covariance matrix of the moment conditions from which a weighting matrix is derived; then used in the second stage to arrive at an efficient estimate of the mean.
I have coded this up in Matlab, and to the best of my knowledge, this has been done correctly. The issue I am having is that the covariance matrix is very close to singular - meaning that it doesn't have an inverse. The 1st stage appears to work ok - the issue is with the second stage not working as required due to this issue.
Any help with diagnosing why I am running into this issue would be most appreciated!
Here is the Matlab code.
clear; close all; clc; %% Generate normal data mu = 10; var = 4; N = 10; X = mu + sqrt(var)*randn(N,1); %% Function definitions for basic GMM f_cost_c = @(mu_hat,degree) (1/N)*sum((X-mu_hat).^degree); % Specify the number of odd moments to include in cost fn N_moments = 3; f_cost_v = @(mu_hat)f_cost_c(mu_hat,1); for i = 1:N_moments-1 f_cost_v = @(mu_hat)[f_cost_v(mu_hat); f_cost_c(mu_hat,2*i+1)]; end f_L_cost_c = @(mu_hat) f_cost_v(mu_hat)'*f_cost_v(mu_hat); %% Find the value of mu_hat which minimises the basic squared cost (the 1st stage of 2S GMM) mu_hat_0 = 5; mu_hat = fminunc(f_L_cost_c,mu_hat_0); mu_hat_1 = mu_hat; %% Use the value of the cost vector at the value found in 1st stage to generate weighting matrix v_cost = f_cost_v(mu_hat); S_hat = (1/N_moments)*(v_cost*v_cost'); W_hat = inv(S_hat); %% 2nd stage of 2SLS GMM % Define new cost using estimated weight matrix f_L2_cost_c = @(mu_hat) f_cost_v(mu_hat)'*W_hat*f_cost_v(mu_hat); mu_hat_0 = mu_hat; mu_hat = fminunc(f_L2_cost_c,mu_hat_0);