It's my understanding that the general form of a variance-covariance matrix has variance terms on the diagonal and covariance terms on the off-diagonal.

I have seen in multiple references (for example Wikipedia) that the maximum likelihood estimator of the variance-covariance matrix for a vector autoregressive model VAR(p) is (note the espilon_t's are vectors):

$$ \hat{\Sigma} = \frac{1}{T} \sum_{t=1}^T \hat{\epsilon}_t \hat{\epsilon}_t^T $$

which I think is the same as (note the y_t's are vectors):

$$ \hat{\Sigma} = \frac{1}{T} \sum_{t=1}^T (y_t - \hat{y}_t)(y_t - \hat{y}_t)^T $$

In that case, though, how could you ever have an off-diagonal entry? I know the model assumes that the covariances are zero, but what if they aren't?


Wait a minute, I think I've done something a little silly. Another way of writing the equation for the variance-covariance matrix is:

$$ \hat{\Sigma} = \frac{1}{T} \hat{\epsilon}^T{\hat{\epsilon}} $$

Suppose for example we have two time steps (T=2) and each residual vector has three entries. Then we have something that looks like this:

$$ \hat{\Sigma} = \frac{1}{T} \begin{bmatrix}\hat{\epsilon}_{11} &\hat{\epsilon}_{21} \\\hat{\epsilon}_{12}&\hat{\epsilon}_{22} \\ \hat{\epsilon}_{13} &\hat{\epsilon}_{23} \end{bmatrix} \begin{bmatrix}\hat{\epsilon}_{11} &\hat{\epsilon}_{12} &\hat{\epsilon}_{13} \\\hat{\epsilon}_{21}&\hat{\epsilon}_{22} &\hat{\epsilon}_{23}\end{bmatrix} $$

The first entry is $$\hat{\Sigma_{11}} = \frac{\hat{\epsilon}_{11}^2 + \hat{\epsilon}_{21}^2}{2}$$

The second entry is $$\hat{\Sigma_{12}} = \frac{\hat{\epsilon}_{11}\hat{\epsilon}_{12} + \hat{\epsilon}_{21} \hat{\epsilon}_{22}}{2}$$


That gives us off-diagonals as required. They might be zero, but they exist. My problem was that I was transposing the wrong vector earlier, and I kept getting scalar outputs instead of matrix outputs. Still, if someone else could confirm that this edit is correct, I'd be grateful.

  • $\begingroup$ Why exactly do you think the off-diagonal entries should be zero? Check out an empirical example and you will see they are not. $\endgroup$ – Richard Hardy Feb 27 at 9:43
  • $\begingroup$ Thanks @RichardHardy, I thought the off-diagonal entries should be zero because of the assumption that the errors are not correlated, but I could be wrong. Regardless, how does this formula yield any off-diagonal entries? $\endgroup$ – StatsSorceress Feb 27 at 15:12
  • $\begingroup$ This may be an assumption but it does not often hold in reality. And if it holds, you get an accurate estimate because you are estimating zero off-diagonal elements as zeros. Instead of explaining how it yields nonzero off-diagonal entries, I can reiterate my question, why would it not? Unless there is a good reason (which I am asking for) for the number to be zero, I would expect all sorts of numbers off the diagonal. $\endgroup$ – Richard Hardy Feb 27 at 15:23
  • $\begingroup$ Thanks @RichardHardy, I guess I'm misunderstanding something fundamental here: I thought this was a TxT matrix. We're plugging the same t into each part of the product in the equation. So we'll only ever get values that belong on the diagonal. How could this formula give an off-diagonal entry? $\endgroup$ – StatsSorceress Feb 27 at 15:25
  • $\begingroup$ Why do you have a 3x3 matrix if you have 2 time periods? It would be something x2 or 2x something, depending on how you transpose it. $\endgroup$ – Richard Hardy Feb 27 at 20:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.