# estimate of variance-covariance matrix for linear models with correlated observations

For linear models $$y_{n \times1}=X_{n \times p}\beta_{p \times 1}+\epsilon_{n \times 1}, \text{ where }\epsilon \sim N(0,V)$$ If in a real life problem we have data as $$(y_1,x_1),(y_2,x_2),...,(y_n,x_n)$$ where $$x_i$$'s are row vectors in $$X$$, and we want to fit the data using the linear model above. Then how could we get a estimate of $$V$$?

I know usually, we are interested in estimating $$\beta$$ and the covariance of $$\beta$$. But here I want to find out the distribution of $$y$$, say $$y \sim N_n(X\beta, V)$$, then what is $$V$$? Is there an estimate of it?

For me, the vector $$y$$ is just one observation from the multinormal distribution $$y \sim N_n(X\beta, V)$$. Like in the univariate normal case, if we have an observation $$z$$ from $$N(\mu, \sigma^2)$$, we can estimate $$\mu$$ as $$\hat{\mu}=\bar{z}=z$$, but we can not estimate $$\sigma^2$$ since we only have one observation. So does that mean we cannot estimate $$V$$ here?

Moreover, if we have a linear mixed effect model $$y=X\beta+Z\alpha+\epsilon$$ where $$\beta$$ is the fixed effect and $$\alpha$$ is the random effects, and $$\epsilon \sim N(0, V) \text{ and }\alpha \sim N(0, \Omega)$$ Then is there a way to estimate $$V$$ and $$\Omega$$? Then we could find the covariance matrix of $$y$$.

Thanks!

In both models, the variance components (i.e., all parameters in the specification of the variance-covariance matrices $$V$$ and $$\Omega$$) are typically estimated using either the Restricted Maximum Likelihood or Maximum Likelihood approaches. These two approaches are, for example, implemented in function lmer() in the lme4 package, and functions gls() and lme() in the nlme package in R.

• Thanks for replying. I use lmer, so they will return a correlation matrix for random effects, which is quite close to cov2cor(Omega), i.e. I will see this as the estimate of $\Omega$. But how could I get $V$? And for the model only has fixed effect, like the first one, I will normally use lm, but I did not see the estimate of $V$ in the summary(lm).
– Nan
Feb 18, 2019 at 15:56
• Or in other words, I do not have to estimate $\Omega$ and $V$ separately in the mixed effect case. I only want to find an estimate of the variance of the response variable $y$.
– Nan
Feb 18, 2019 at 16:02
• The lme4 package works under the random effects paradigm and cannot directly estimate a model with correlated error terms. This is available in the lme() function from the nlme package. From a mixed model fitted by lme(), you can get the covariance matrix of the marginal model for $y$ using the getVarCov() function. Feb 18, 2019 at 16:23
• Thanks. Do you have any ideas under the fixed effects model like the first model I gave? I use lm, but they only give the residuals for each observation, i.e. residual=observation-fitted value, and this is not what I want.
– Nan
Feb 18, 2019 at 16:35
• For the first model you will need to use function gls() from the nlme package and suitably specify its correlation and weights arguments. Feb 18, 2019 at 18:09