# Confidence interval for covariance in multivariate linear regression

Let $$X\in R^{n\times q}$$ and $$Y\in R^{n\times p}$$ be our data matrices, and we assume that they are related by a linear model $$Y = BX + \Xi$$, where $$B$$ is fixed and unknown and each row $$\Xi_i \sim \mathcal{N}(0, \Sigma)$$ is iid drawn from a multivariate Gaussian with fixed but unknown covariance matrix $$\Sigma$$.

We want to estimate the covariance of the two dependent variables, controlled for the independent variable. If I am correct, this is represented by the off-diagonal elements of $$\Sigma$$.

We can estimate $$\Sigma$$ as $$\hat \Sigma = \frac{1}{n-q-1}(Y-X\hat B)^T(Y-X\hat B)$$, where $$\hat B = (X^T X)^{-1}X^T Y$$.

But I want to double check how confident I should be in this estimate, ie I want to compute a confidence interval around the off-diagonal parameter of $$\hat \Sigma$$.

How do I do that?

• Adrian Hutter suggested me to look into bootstrapping. I am not sure if I want to rely on this though because my sample size is very small (~30 samples) Feb 18, 2020 at 14:17

If the sample size really is 30, like in the comment, the bootstrap might still be OK, but if one really thinks the sample is too small for a bootstrap, then an analytical solution isn't going to help you, because it's going to rely on the assumption of normal errors which probably isn't justified. That's my intuition at least. So one could spend time thinking about an analytical solution but I don't feel it'd actually be a good idea. You could run simulations with reasonable values of $$n,X,B,\Sigma$$ given your problem and see how the bootstrap performs.
Minor corrections: I think you want $$Y = XB + \Xi$$, and that we want to estimate the covariance of any pair of the $$p$$ dependent variables (or are considering the $$p = 2$$ case). The formula for $$\hat{\Sigma}$$ can have $$1 / (n - q)$$ instead of $$1 / (n - q - 1)$$ if we assume $$X$$ already includes any desired intercept (as the formulas indicate).