I need to create an upper and lower bound for the error variance, in linear regression or otherwise (state space models etc.). One way is to bootstrap confidence intervals, but that can be very computationally expensive, especially in state space models.

Is there a measure to estimate the bounds for error variance because I do not need the whole distribution.

  • $\begingroup$ For normal data pivot $\frac{(n-1)S^2}{\sigma^2}\sim\mathsf{Chisq}(\nu=n-1).$ $\endgroup$
    – BruceET
    Oct 16 at 17:41

Pivoting $\frac{(n-1)S^2}{\sigma^2}\sim\mathsf{Chisq}(\nu=n-1),$ as suggested in comment gives a CI for $\sigma^2$ of the variance $\sigma^2$ of the normal population from which data were randomly sampled.

A 95% Ci is $\left(\frac{(n-1)S^2}{U}, \,\frac{(n-1)S^2}{L}\right),$ where $L$ and $U$ cut probability $0.025$ from lower and upper tails, respectively, of $\mathsf{Chisq}(n-1).$ Take square roots of the endpoints to get the corresponding CI for $\sigma.$ In R:

x = rnorm(100, 50, 7)               # 7 is population SD
[1] 51.84549  # sample variance
99*var(x)/qchisq(c(.975,.025), 99)
[1] 39.96748 69.96494               # 95% CI for pop var

99*var(x)/qchisq(c(.995,.005), 99)
[1] 36.92944 77.17179               # 99% CI for pop var
sqrt(99*var(x)/qchisq(c(.995,.005), 99))
[1] 6.076960 8.784747               # 99% CI for pop SD

So a 95% CI for $\sigma^2$ is $(30.97, 60.96),$ containing $\sigma^2 = 49,$ as should be the case in 95% of such demonstrations.

Note: Bootstrapping may be the only reasonable option if data are from a population with an unknown type of distribution. Even then you need to feel comfortable with the assumption that the population has a finite variance.

Pretending that the population is of unknown distribution type, the following style of bootstrap 95% CI $(39.66, \,63.81)$ for $\sigma^2$ is noticeably shorter than the 95% CI shown above based on the chi-squared distribution.

v.obs = var(x)
d.re = replicate(2000, var(sample(x,100,rep=T))-v.obs)
LU = quantile(d.re, c(.975,.025))
v.obs - LU
   97.5%     2.5% 
39.66229 63.81144 

A bootstrap CI using ratios instead of differences (because $\sigma^2$ has more to do with scale than with location) may be preferable:

v.obs = var(x)
r.re = replicate(2000, var(sample(x,100,rep=T))/v.obs)
LU = quantile(r.re, c(.975,.025))
v.obs / LU
   97.5%     2.5% 
42.01865 66.62621 

In both cases, the bootstrap CI is shorter than the CI based the assumption that the population is normal. The assumption of normality envisions values in both directions beyond what the sample shows:

   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
  34.21   43.10   49.06   48.78   53.42   64.84 
[1] 51.84549

By contrast, the nonparametric bootstraps envision only population values between the observed min and max--34.21 and 64.84 for my fictitious data above.

If we forgot, or never knew, the CI above using a chi-squared distribution, we might use a parametric bootstrap CI. A parametric bootstrap repeatedly re-samples from a normal population with estimated mean and variance. The nonparametric bootstrap below, using an elementary quantile approach, is one possibility.

v.re = replicate( 5000, var(rnorm(100,mean(x),sd(x))) )
quantile(v.re, c(.025,.975))
    2.5%    97.5% 
38.25340 66.82498 

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.