I am new to method of moments and want to figure out how to derive the method of moment estimator for $\beta$ in the linear equation with a constant term and three corresponding moments, namely, I have

$$y_i = \alpha+ x_i \beta + u_i$$

And I have the following moments: $E[x_i - \mu] = 0$, $E[y_i - \alpha+ x_i \beta_1] = 0$ and $E[x_i(y_i - \alpha+ x_i \beta)] = 0$

Thanks in advance


Using the methods of moment, you need to compare the sample statistics and population statistics.

For a classical linear regression, your residue $u_i \sim N(0, \sigma^2)$ and there should no correlation between residue $u_i$ and your feature vector $x_i$ as you mentioned in the problem statement(strict exogeneity). Then MoM gives the following equations

$$E(u) = 0 = \frac{1}{N}\sum_{i=1}^n (y_i-\beta x_i), (1)$$

$$E(x(y-\beta x)) = \mathbf{0} = \frac{1}{N}\sum_{i=1}^n x_i(y_i-\beta x_i), (2)$$

Keep in mind the second formula is p-dim.

You do not have to look at higher-order moments ($E[(y_i-\beta x_i)^2] = \sigma^2$) as they are not useful for deriving the closed form in linear regression.

The most straight forward way is to convert the formula (2) into a matrix product form $$ X^T(Y-X\beta) = \mathbf{0}$$ Then you have the standard OLS estimator $$\hat{\beta} = (X^TX)^{-1}X^Ty $$ when $rank(X) = n$.

P.S, with MoM, it is easier to treat data with heteroscedascity, or X-column correlation because you can do the second-order estimation.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.