Variance-Covariance matrix interpretation

Question

Assume we have a linear model Model1 and vcov(Model1) gives the following matrix:

             (Intercept)    latitude  sea.distance   altitude
(Intercept)    28.898100 -23.6439000  -34.1523000  0.50790600
latitude      -23.643900  19.7032500   28.4602500 -0.42471450
sea.distance  -34.152300  28.4602500   42.4714500 -0.62612550
altitude        0.507906  -0.4247145   -0.6261255  0.00928242

For this example, what does this matrix actually display? What assumptions we can safely make for our model and it's independent variables?

Answer 1

This matrix displays estimates of the variance and covariance between the regression coefficients. In particular, for your design matrix $\mathbf{X}$, and an estimate of the variance, $\widehat{\sigma}^2$, your displayed matrix is $\widehat{\sigma}^2(\mathbf{X}'\mathbf{X})^{-1}$.

The diagonal entries are the variance of the regression coefficients and the off-diagonals are the covariance between the corresponding regression coefficients.

As far as assumptions go, apply the cov2cor() function to your variance-covariance matrix. This function will convert the given matrix to a correlation matrix. You wil get estimates of the correlations between the regression coefficients. Hint: for this matrix, each of the correlations will have large magnitudes.

To say something about the model in particular, we need point estimates of the regression coefficients to say anything further.

Answer 2

@Donnie has provided a good answer (+1). Let me add a couple points.

Running down the main diagonal of the variance-covariance matrix are the variances of the sampling distributions of your parameter estimates (i.e., $\hat\beta_j$'s). Thus, taking the square roots of those values yields the standard errors that are reported with statistical output:

SEs   = sqrt(diag(vcov(Model1)))
SEs
# [1] 5.37569530 4.43883431 6.51701235 0.09634532

These are used to form confidence intervals and test hypotheses about your betas.

The off-diagonal elements would be $0$ if all variables were orthogonal, but your values are far from $0$. Using the cov2cor() function, or standardizing the covariances by the square roots of the constituent variable variances reveals that all variables are highly correlated ($|r| > .97$), so you have substantial multicollinearity. This makes your standard errors much larger than they would otherwise be. Likewise, it means that there is a great deal of information about the sampling distributions of the betas that is being left out of standard hypothesis tests ($\hat\beta_j/SE(\hat\beta_j)$), so you may want to use a sequential testing strategy based on type I sums of squares.