# GMM Estimator of an Exponential Distribution

Suppose you have to calculate the GMM Estimator for $\lambda$ of a random variable with an exponential distribution.

$$f(x) = \lambda \cdot \exp(-\lambda\cdot x)$$ with $E(X) = 1/\lambda$ and $E(X^2) = 2/\lambda^2$.

Wouldn't the GMM and therefore the moment estimator for $\lambda$ simply obtain as the sample mean to the power of minus 1?

• Is this self-study? Please add the tag if so and read its wiki. – Christoph Hanck Mar 21 '16 at 13:00
• I worked on your typesetting. Please check everything is still as desired – Christoph Hanck Mar 21 '16 at 14:24
• Yes everything seems to be in order. – narain Mar 21 '16 at 14:35
• No, $\bar{X}^{-1}$ would just be MM based on the first moment condition. – Christoph Hanck Mar 21 '16 at 14:44
• Also, it is not true that $E(X^2)=1/\lambda^2$. – Christoph Hanck Mar 21 '16 at 15:10

The misunderstanding here is that GMM exploits both moment conditions simultaneously. As there are more ($=2$) moment conditions than unknown parameters ($=1$), there is no value that uniquely solves both moment equations $$E(X)-1/\lambda=0$$ and $$E(X^2)-2/\lambda^2=0$$ GMM therefore minimizes the weighted squared difference between the empirical version of the moments and the functions of the parameters, weighted by some suitable (positive definite) weighting matrix.

Thus, let $\bar{X}$ the sample average and $\bar{X^2}=\frac{1}{n}\sum_iX_i^2$. Next, let

$$m(\lambda)=\begin{pmatrix}\bar{X}-1/\lambda\\\bar{X^2}-2/\lambda^2\end{pmatrix}$$ Taking the identity matrix as the weighting matrix for simplicity (see below for a more efficient alternative), the GMM minimization problem becomes $$\min_\lambda m(\lambda)'m(\lambda)$$ A bit of algebra will give you a FOC $$\bar{X}\lambda^3+(4\bar{X^2}-1)\lambda^2-8=0$$ In R, you could solve that as follows:

polyroot(c(-8,0,4*xbar2-1,xbar))


Let us try some data:

n <- 100000
x <- rexp(n)

xbar <- mean(x)
xbar2 <- mean(x^2)


The admissible (positive) solution seems to do the trick (note the default is $\lambda=1$)

> polyroot(c(-8,0,4*xbar2-1,xbar))
[1]  0.9993932-0i -1.1701779-0i -6.8476177+0i


It seems worth emphasizing, however, that GMM is not efficient here, as the MLE $1/\bar{X}$ already is. Consider this little Monte Carlo simulation:

reps <- 100000

MLE <- GMM <- rep(NA,reps)

for (i in 1:reps){
x <- rexp(n)

xbar <- mean(x)
xbar2 <- mean(x^2)

MLE[i] <- 1/xbar
GMM[i] <- max(Re(polyroot(c(-8,0,4*xbar2-1,xbar))))
}
plot(density(MLE),lwd=2,col="purple")
lines(density(GMM),lwd=2,col="lightblue")
abline(v=1,lty=2)


The following plot shows that ML is not only much simpler, but more efficient:

A more efficient GMM estimator is obtained by employing an efficient weighting matrix, i.e., one that converges to the inverse of the variance matrix of the moment conditions:

optGMM <- function(lambda,x){
xbar <- mean(x)
xbar2 <- mean(x^2)
preliminaryGMM <- max(Re(polyroot(c(-8,0,4*xbar2-1,xbar))))

residuals1 <- x - 1/preliminaryGMM
residuals2 <- x^2 - 2/preliminaryGMM^2
S11 <- mean(residuals1^2)
S22 <- mean(residuals2^2)
S12 <- mean(residuals1*residuals2)
S <- matrix(c(S11,S12,S12,S22),ncol=2)
m <- c(xbar-1/lambda,xbar2-2/lambda^2)
t(m)%*%solve(S)%*%m
}
optimise(optGMM,interval=c(0,1e5),x=x)$minimum  Including this in the simulation (for$n=1000\$ now) gives

The variance of this distribution is only marginally higher than that of the MLE in the simulation.