I have been reading about the method of moments, and now I understand how to obtain the method of moments estimator for a random sample $x_1,...,x_n$ from a distribution $f(x;\theta)$, in the multiparameter case. However, I fail to see how to formulate the method of moments for the linear regression model $$y_i = x_i^T\beta + e_i,$$ where $E[e_i]=0$, $\beta,xi\in R^p$. There is this moment condition, but I cannot link it to a system of equations as in least squares estimation.

Is there a reference with this formulation?

  • $\begingroup$ To derive a moment condition, you can use the first order condition from the least squares optimization problem, or the orthogonal projection theorem. $\endgroup$ – Frank Nov 9 '18 at 14:22

The least squares estimator is the solution to the estimating equation:

$$ 0 = \mathbf{X}^T \left( Y - \mathbf{X}\beta \right)$$

Where $\mathbf{X} = [1, x_1, x_2, \ldots, x_p]$ is a $n \times p$ model matrix of covariate(s).

This is a trivial result, but a more general discussion on estimating equations can be found in Wakefield "Bayesian and Frequentist Regression Methods". Estimating Equations are also called M-estimators. Another reference would be Boos, Stefanski "Essential Statistical Inference" ch. 7.

| cite | improve this answer | |
  • $\begingroup$ I would add the qualitative comment that this expresses/enforces a condition of orthogonality between (population) residuals and explanatory variables which says "there is nothing omitted from the model that is correlated to something that has been included". $\endgroup$ – user8948 Nov 12 '18 at 13:56

@AdamO indicates the solution using OLS, which you seem to know already. In order to obtain the desired formulation, you need to use the Generalised Method of Moments, in order to facilitate this formulation.


The idea is to start with the equation

$${\displaystyle \operatorname {E} [\,x_{t}(y_{t}-x_{t}^{\mathsf {T}}\beta )\,]=0}$$

and produce the equivalent sample equation.

| cite | improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.