Generalized Least Squares: Estimation of Variance-Covariance matrix Linear model in matrix form is
$
\mathbf{y}=\mathbf{X}\beta+\epsilon\textrm{ where }\epsilon\sim\mathbb{N}\left(0,\sigma^{2}\mathbf{V}\right).
$
Then $\beta$ can be estimated through generalized least squares
as
$
\widetilde{\beta}=\left(\mathbf{X}^{\prime}\mathbf{V}^{-1}\mathbf{X}\right)\mathbf{X}^{\prime}\mathbf{V}^{-1}\mathbf{y}.
$
I'm wondering how to estimate $\mathbf{V}$ here. I'd highly appreciate
if you give me some pointer or give a reference for reading. Thanks
 A: From what you've written, it appears that you know $\mathbf V$ already. Usually, if you write your covariance matrix as $\sigma^2 \mathbf V$, it means that you know the structure given by $\mathbf V$ (e.g., exchangeable for cluster correlations, or something like that), and you only need to estimate the scale $\sigma^2$.
If you don't know $\sigma^2\mathbf V$, but (i) have a good idea about the structure that $\mathbf V$ should have, and (ii) expect the effects to be pretty strong (e.g., difference in variance for different $y_i$'s by a factor of 5, and/or correlations greater than 0.5), then you can proceed via the feasible GLS path (a random econometrics handout from the first page of Google): estimate your model by OLS, construct an estimate $\hat {\mathbf V}$, and plug that estimate in. To do that though does require some sort of a model for $\mathbf V$ so as to be able to describe it with at most a handful of parameters (e.g., correlation coefficient for exchangeable structure; an autoregression correlation or two for time series; a smoothed model for variance with heteroskedastic but uncorrelated errors; etc.). No good at all will come from forming $\hat{\mathbf V}={\mathbf e}{\mathbf e'}$, as this matrix won't even be invertible.
If you don't know much about this covariance at all, and potentially it may not affect results much, and may not provide any worthwhile efficiency gains, you would be better off leaving things as they are: if you are overdoing any sort of corrections on i.i.d. data, you are just unnecessarily increasing the variance of the estimates.
Amemiya's Advanced Econometrics treats the subject of generalized least squares and FGLS in and out.
