4
$\begingroup$

Suppose that one has the usual regression model $\mathbf{y} = \mathbf{X} \boldsymbol{\beta} + \boldsymbol{\varepsilon}$, where each $\varepsilon_t$ is iid distributed with $\mathbb{E}(\varepsilon_t) = 0$ and $\mathbb{V}\text{ar}(\varepsilon_t) = \sigma^2$. Assume also that variables in $\mathbf{X}$ are non-stochastic for simplicity. It is well-known that asymptotically $$T:=\frac{(n-k)s^2}{\sigma^2} \sim \chi_{n-k},$$ where $s^2 := 1 / (n-k) \sum_t e_t^2$, and $e_t$ denotes least-squares residuals, $n$ is the sample size, and $k$ is the number of columns of $\mathbf{X}$.

Usually when one wants to construct a confidence interval for some $\beta_j$, one uses the so-called studentized bootstrap confidence interbal, which makes use of the $t$-statistic (see, e.g, Davison and Hinkley for details). Is it possible to improve upon the regular percentile bootstrap for the variance $\sigma^2$ by making use of the above statistic $T$? Namely for each $i = 1, \dots, B$, resample the data in some way, compute the statistic $\hat{T}_{(i)} = (n-k) s^2_{(i)}/s^2$, where $s^2$ is the original estimate, order the $\hat{T}_{(i)}$'s, and then construct the interval, as in the standard case.

$\endgroup$

1 Answer 1

2
$\begingroup$

Bootstrap t confidence intervals are not normally used in OLS regression. They can be used when the noise term is very non-Gaussian. For variance estimation with heavytailed distributions all bootstrap confidence intervals have slow convergence. The higher order bootstraps like bootstrap t and BCa converge faster but are not always superior to the percentile method in small samples. See my paper with Robert LaBudde referenced here in this WIRE article.: http://statprob.com/encyclopedia/ResamplingMethods.html

$\endgroup$
4
  • $\begingroup$ Thanks! What do you mean by saying that bootstrap t CIs are not normally used in OLS though? Some authors (e.g. Davidson & MacKinnon) argue that this is the preferred method. $\endgroup$
    – weez13
    Jul 31, 2012 at 8:32
  • $\begingroup$ If the normality assumptions hold parametric confidence intervals should be available for estimating variance components in the linear model. The asymptotic chi-square statistic that you mentioned is used and under normality assumptions it is exact. $\endgroup$ Jul 31, 2012 at 10:01
  • $\begingroup$ Sure, but I suppose the reason for bootstrapping anything is that the error terms are not likely to be normally distributed, or that there is some other complication, e.g. lagged dependent variables in $\mathbf{X}$ matrix? $\endgroup$
    – weez13
    Jul 31, 2012 at 13:19
  • 2
    $\begingroup$ In practice distributions will not be exactly normal. That does not mean that we should always trow out the parametric method and use resampling. Other complications like lag dependencecan be handled with more complex parametric models when they are appropriate. Use the bootstrap when you need it but don't apply it indiscriminantly. $\endgroup$ Jul 31, 2012 at 13:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.