# Negative values for OLS variance

I am currently writing some code which performs regression and have noticed that when I calculate variance of $$c\hat{\beta}$$ I am sometimes on some datasets getting negative values.

The variance is given as $$c'(X'X)^{-1}c\frac{e'e}{n - p}$$ where $$c$$ is a vector representing the contrast to be tested, $$X$$ is the design matrix, $$e$$ is a vector of residuals and $$n$$ is the number of datapoints and $$p$$ is the number of parameters (including an intercept).

I want to compute a T statistic but this involves square rooting the variance and is failing in some cases when the variance is negative. My question is could there be a mathematical/statistical reason why I am having this issue? And if so why/why not? (or should I be asking this instead on StackOverflow?)

My datasets vary in size and negative values appear to occur the most frequently when $$n$$ is only just greater than $$p$$ (e.g. when $$n=p+1,n=p+2,...$$etc ).

• My guess is that when, $n\approx p$ invertability of matrix becomes numerically unstable due to your choice of how to evaluate the expression for the variance, have you inspected the behavior of your matrix inversions? Any warnings? – Jesper Hybel Jan 2 at 18:51
• My question would be, why are you writing your own code to do this? Nearly every modern language has statistical packages capable of performing regression. – StatsStudent Jan 2 at 18:59
• Are you calculating a generalized inverse? Also, are you rounding your results? – StatsStudent Jan 2 at 19:00
• My bet is that you are not calculating a generalized inverse and if $(X'X)$ your matrix is singular and your program is outputting -1 or some other garbage answer. Are you able to share your code or data? – StatsStudent Jan 2 at 19:14
• @Carl Experience teaches that there are many other ways to compute a negative quantity for a variance. Some variances are computed as differences of two positive values and floating point error is enough to create a negative result in the computer. The comments preceding yours suggest plausible mechanisms for the formula in the question to produce negative values, depending on how the calculation is performed. – whuber Jan 2 at 21:17