2
$\begingroup$

Suppose you have to calculate the GMM Estimator for $\lambda$ of a random variable with an exponential distribution.

$$f(x) = \lambda \cdot \exp(-\lambda\cdot x)$$ with $E(X) = 1/\lambda$ and $E(X^2) = 2/\lambda^2$.

Wouldn't the GMM and therefore the moment estimator for $\lambda$ simply obtain as the sample mean to the power of minus 1?

$\endgroup$
  • $\begingroup$ Is this self-study? Please add the tag if so and read its wiki. $\endgroup$ – Christoph Hanck Mar 21 '16 at 13:00
  • $\begingroup$ I worked on your typesetting. Please check everything is still as desired $\endgroup$ – Christoph Hanck Mar 21 '16 at 14:24
  • $\begingroup$ Yes everything seems to be in order. $\endgroup$ – narain Mar 21 '16 at 14:35
  • $\begingroup$ No, $\bar{X}^{-1}$ would just be MM based on the first moment condition. $\endgroup$ – Christoph Hanck Mar 21 '16 at 14:44
  • $\begingroup$ Also, it is not true that $E(X^2)=1/\lambda^2$. $\endgroup$ – Christoph Hanck Mar 21 '16 at 15:10
2
$\begingroup$

The misunderstanding here is that GMM exploits both moment conditions simultaneously. As there are more ($=2$) moment conditions than unknown parameters ($=1$), there is no value that uniquely solves both moment equations $$ E(X)-1/\lambda=0 $$ and $$ E(X^2)-2/\lambda^2=0 $$ GMM therefore minimizes the weighted squared difference between the empirical version of the moments and the functions of the parameters, weighted by some suitable (positive definite) weighting matrix.

Thus, let $\bar{X}$ the sample average and $\bar{X^2}=\frac{1}{n}\sum_iX_i^2$. Next, let

$$ m(\lambda)=\begin{pmatrix}\bar{X}-1/\lambda\\\bar{X^2}-2/\lambda^2\end{pmatrix} $$ Taking the identity matrix as the weighting matrix for simplicity (see below for a more efficient alternative), the GMM minimization problem becomes $$ \min_\lambda m(\lambda)'m(\lambda) $$ A bit of algebra will give you a FOC $$ \bar{X}\lambda^3+(4\bar{X^2}-1)\lambda^2-8=0 $$ In R, you could solve that as follows:

polyroot(c(-8,0,4*xbar2-1,xbar))

Let us try some data:

n <- 100000
x <- rexp(n)

xbar <- mean(x)
xbar2 <- mean(x^2)

The admissible (positive) solution seems to do the trick (note the default is $\lambda=1$)

> polyroot(c(-8,0,4*xbar2-1,xbar))
[1]  0.9993932-0i -1.1701779-0i -6.8476177+0i

It seems worth emphasizing, however, that GMM is not efficient here, as the MLE $1/\bar{X}$ already is. Consider this little Monte Carlo simulation:

reps <- 100000

MLE <- GMM <- rep(NA,reps)

for (i in 1:reps){
  x <- rexp(n)

  xbar <- mean(x)
  xbar2 <- mean(x^2)

  MLE[i] <- 1/xbar
  GMM[i] <- max(Re(polyroot(c(-8,0,4*xbar2-1,xbar))))
}
plot(density(MLE),lwd=2,col="purple")
lines(density(GMM),lwd=2,col="lightblue")
abline(v=1,lty=2)

The following plot shows that ML is not only much simpler, but more efficient:

enter image description here

A more efficient GMM estimator is obtained by employing an efficient weighting matrix, i.e., one that converges to the inverse of the variance matrix of the moment conditions:

optGMM <- function(lambda,x){
  xbar <- mean(x)
  xbar2 <- mean(x^2)
  preliminaryGMM <- max(Re(polyroot(c(-8,0,4*xbar2-1,xbar))))

  residuals1 <- x - 1/preliminaryGMM
  residuals2 <- x^2 - 2/preliminaryGMM^2
  S11 <- mean(residuals1^2)
  S22 <- mean(residuals2^2)
  S12 <- mean(residuals1*residuals2)
  S <- matrix(c(S11,S12,S12,S22),ncol=2)
  m <- c(xbar-1/lambda,xbar2-2/lambda^2)
  t(m)%*%solve(S)%*%m
}
optimise(optGMM,interval=c(0,1e5),x=x)$minimum

Including this in the simulation (for $n=1000$ now) gives

enter image description here

The variance of this distribution is only marginally higher than that of the MLE in the simulation.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.