4
$\begingroup$

For a simple linear model such as $y_i = X_i\beta + \epsilon $ for i = 1,2...

and we assume that $E[\epsilon_i | X] = 0$ and that $E[\epsilon_i^2|X] = \sigma_i^2 I_T$ for i=1,2...,n and that $E[\epsilon_i,\epsilon_j'] = \sigma_{ij} I_T$ for $i \neq j$

Can I use MLE to estimate this model? Will the correlation between errors cause problems with the standard errors of the MLE estimates of $\beta$?

Improve this question
$\endgroup$
3
  • $\begingroup$ I think your question a little bit confusing, for a simple linear regression $E[\epsilon_i,\epsilon_j'] = 0$ for $i \neq j$, then MLE and OLS are the same. If $E[\epsilon_i,\epsilon_j'] \neq 0$ you may consider REML. $\endgroup$
    – Deep North
    Commented Nov 24, 2017 at 4:21
  • $\begingroup$ The OP is setting up the problem with a known correlation structure for the error terms. But if the variances and covariances are to be estimated that is a lot of terms to estimate and won't even be estimable if the sample size not sufficiently large. Certainly estimating so many error terms will lead to a larger standard error for the estimate of $\beta$. $\endgroup$
    – Michael R. Chernick
    Commented Nov 24, 2017 at 15:23
  • $\begingroup$ Do you actually assume a panel-data like structure where observations vary along two dimensions, so that you need $n$ and $T$, or is $n=T$? In the first case, it does not seem to be appropriate to call it the "simple linear model". In the second, we have a standard GLS problem, it seems. $\endgroup$
    – Christoph Hanck
    Commented Nov 24, 2017 at 15:40

2 Answers 2

Reset to default
7
$\begingroup$

The error (correlation) structure you assume simply needs to be incorporated in the likelihood function that you are trying to maximize. Remember that the logarithmic Gaussian likelihood function is generally written $$ \ell(\mathbf{\beta},\mathbf{\Sigma}) = -\frac{n}{2} \log ( 2 \pi) - \frac{1}{2} \log \left| \mathbf{\Sigma} \right| -\frac{1}{2}\left( \mathbf{y} - \mathbf{X} \mathbf{\beta} \right)^{\mathsf{T}} \mathbf{\Sigma}^{-1} \left( \mathbf{y} - \mathbf{X} \mathbf{\beta} \right) $$ where in the homoskedastic model we assume spherical errors, i.e. $\mathbf{\Sigma} = \sigma^{2} \mathbf{I}_{n}$, such that the variance-covariance matrix of the error has a constant $\sigma^{2}$ on the diagonal and zeros off-diagonally. But nothing prevents us from assuming (and estimating) a more complex variance-covariance matrix, including heteroskedasticity and/or serial correlation.

Improve this answer
$\endgroup$
0
0
$\begingroup$

Yes, the solution is to use weighted least squares. The correlation structure is exchangeable, if I understand your notation. Generalized least squares simultaneously estimate the linear model and correlation structure with the EM algorithm.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.