I am currently trying to become familiar with design of experiments with the book “Optimal design of experiments” by Goos and Jones. In chapter 2, they discuss the use of a Plackett–Burman type design (6 factors, 12 experiments), and in chapter 2.3 (“Peek into the black box”), they introduce the math. I am currently struggling with chapter 2.3.4 (“Ordinary least squares estimates”), more specifically with some of the equations.
Quote:
The ordinary least squares estimator of the vector of unknown model >coefficients $\beta $ is $$\hat{\beta }=({X}'X)^{-1}{X}'Y.$$
Ok, I understand that, assuming X' is the transposed of X. Then they continue
The variance-covariance matrix of this estimator is $$var(\hat{\beta })=\sigma _{\varepsilon } ^{2}({X}'X)^{-1}.$$
Here, I am getting confused. I thought that $\beta$ (the "real" values) and $\hat{\beta}$ (the calculated values) are vectors, and that the elements $\beta _i$ are scalars. However, the equation and the wording suggest that $\hat{\beta}$ is a matrix, and that the elements $\hat{\beta}_i$ are vectors. Moreover, how is it possible to calculate anything related to $\beta$ without the regressand vector Y. What am I missing here?
Further down in the text, they write
Note that the variance-covariance matrix of the estimator is directly proportional to the error variance, which is unknown. We can estimate the error variance using the mean squared error: $$\hat{\sigma} > _{\varepsilon } ^{2} = \frac{1}{n-p}(Y- X \hat{\beta} )' (Y- X \hat{\beta} ).$$
I can understand that equation. At least, it makes sense that the difference between Y and $X\hat{\beta }$ (or $\hat{Y}$) is proportional to $\hat{\sigma} _{\varepsilon }$.
But I would really like to understand the other equations as well.
Regards, Soltub