Is independence assumption needed for Method of Moments estimator

Question

I read about Method of Moments estimator (MOM) in Statistical Inference by Casella and Berger. In MOM description, I do not see the requirements that the sample should be iid. However, in the examples, there is an explicit assumption about the iid data.

As far as I understand, to compute MOM estimate, I need to equate the sample moments with the corresponding population moments and solve the resulting equations. It is not obvious to me that I need independent observations to compute that.

Could you please clarify if, in general, MOM estimator requires independent data points? What can go wrong if the observations are not independent?

Answer 1

Linearity of expectation doesn't rely on independence.

Consequently you don't require independence for $E(\bar{X})$ to equal $\mu$ (similarly for higher moments). So we just need that the expectation of $\overline{X^k}$ exists for it to equal $\mu^\prime_k$.

However if you want consistency (a property many people will expect in an estimator), you'll need that they're not "too dependent" (consider a sequence of perfectly correlated identically distributed variables, $X_1=X_2=X_3=...$ -- the variance of the average is the same as the variance of a single value -- you definitely don't have consistency there).

[There are, for example, versions of the weak law of large numbers for dependent variables with particular kinds of bounds on the covariances. (e.g. see this math.SE question).]

---

Now consider a couple of examples of sets of dependent variables to see that in some cases it will work okay and in others -- even with no variables being highly correlated -- it doesn't:

1. Take $Z\sim (0,\sigma^2), X_i\sim (0,\sigma^2)\,,\quad i=1,2,...$ (where all variables are independent with the given mean and variance, and the $X_i$ are identically distributed)

   Now let $Y_i=\mu + kZ + \sqrt{1-k^2}\,X_i$ for some specific $0<k<1$.

   Then $E(Y_i)=\mu$ and $\text{Var}(Y_i) = \sigma^2$.

   \begin{eqnarray} \text{Cov}(Y_i,Y_j) &=& E[(Y_i-\mu)(Y_j-\mu)]\\ &=& E[(kZ + \sqrt{1-k^2}\,X_i)(kZ + \sqrt{1-k^2}\,X_j)]\\ &=&k^2E(Z^2)=k^2\sigma^2\,, \end{eqnarray}

   and the correlation between any pair of $Y$-variables is therefore $k$.

   $\text{Var}(Y_1+Y_2+...+Y_n) = (n\sigma^2 + n(n-1) k \sigma^2) $

   $\text{Var}(\bar{Y}) = \sigma^2(1/n + (n-1)/n k) > k\sigma^2 $

   Even though we have that $\bar{Y}$ is an unbiased estimator of $\mu$, this would not be particularly attractive because $\bar{Y}$ approaches $\mu+kZ$ -- even with a small common correlation, this estimate doesn't really improve -- once you add enough variables into the average to make the contributions of the $X$'s small, adding more variables in doesn't really help.

2. Now consider another set of dependent variates:

   $Y_1=\mu+X_1\,,\quad$ (where the $X_i$ are defined as above)

   $Y_i=\mu+k(Y_{i-1}-\mu)+X_i\,,\quad i=2,3,...$ (again for some particular $0<k<1$).

   Now it turns out that we have that $E(\bar{Y})=\mu$ and $\sqrt{n}(\bar{Y}-\mu)$ has a finite variance (essentially because the correlations decrease geometrically in $|i-j|$, ultimately making it so the variance of the sum eventually grows only proportionally to $n$), so here $\bar{Y}$ makes more sense as an estimator of $\mu$ even though these variables are all correlated.

Answer 2

As was pointed out in the comments, identically distributed data is required, otherwise, the estimate does not make sense.

As for the independence assumption, I guess that the main reason for it is the requirement imposed by the law of large numbers. Without iid, there is no guarantee that MOM estimate converges to the true mean.