Upper bound for variance of $\hat{\beta}$ in multiple linear regression

Question

The variance of the beta estimator in an ordinary-least-squares multiple linear regression to express $Y$ as a (linear) function of $X$, $\hat{\beta}$, can be expressed as (knowing $X$ and $\sigma^2$ the variance of the residuals, provided the Gauss-Markov assumptions hold):

$Var[\hat{\beta}|X] = \sigma^2 (X^TX)^{-1}$

Starting from this point, is there a known (and as tight as possible) upper bound (in $\mathbb{R}$) for all $Var[\hat{\beta_j}|X] = \sigma^2 (X^TX)_{jj}^{-1}$ with $j > 0$ (the constant coefficient of the linear regression being out of the scope)?

For sure the upper bound will depend on the regression data, like $\sigma^2$, $\sigma_X^2$, $\sigma_Y^2$. But this upper bound would be in $\mathbb{R}$, while the matrix form can be complicated to comprehend in the most general case.

If such an upper bound is not known, does there exist an expression for each $\hat{\beta_j}$ that is not in matrix form, and from which an upper bound can be more easily derived?

Answer 1

There is no non-trivial upper bound that doesn't depend on a lot of detail about the matrix.

A conceptually simple version of the variance is $$\mathrm{var}[\hat\beta]_{jj}=\frac{\mathrm{var}[Y|X]}{n\mathrm{var}[X_j|\textrm{other Xs}]}$$

Even fixing the distribution of each $X$ separately and of $Y$, the denominator can be an arbitrarily small positive number or zero, so the variance can be an arbitrarily large finite number or infinite.

Bounding the individual covariance terms is also not sufficient to get you anywhere: when the number of variables $p$ is large it is possible to get a zero denominator with all the covariances being small (about $-1/p$).

Since $\mathrm{var}[X_j|\textrm{other Xs}]\leq \mathrm{var}[X_j]$, a lower bound on the variance of $\beta_j$ is available from the separate distributions of $Y$ and the $X$s.