6
$\begingroup$

Background

I'm currently looking into how reliable confidence intervals are on bounded scores (EQ-5D) and I compare regular asymptotic, robust and botstrap-based confidence intervals. The robust (alias heteroscedacity-consistent) perform really well but have a significant over-coverage in small samples. This gets better if I use the normal distribution instead of the t-distribution, unfortunately I'm finding conflicting data on what distribution to use.

Question

Is there a general recommendation for using the t-distribution instead of the normal distribution? (A reference would be lovely)

Details

I'm using the sandwich package in R together with "HC3" & "HC4m" covariance estimators generated by the vcovHC function. I started using th code from StackOverflow's question on this subject, after adapting it a little I have the following code:

confint.robust <- function(object, parm, level = 0.95, HC_type="HC3", ...){
  cf <- coef(object); pnames <- names(cf)
  if(missing(parm))
    parm <- pnames
  else if (is.numeric(parm))
    parm <- pnames[parm]

  a <- (1-level)/2; a <- c(a, 1-a)
  pct <- stats:::format.perc(a, 3)
  fac <- qt(a, object$df.residual)
  ci <- array(NA, 
              dim = c(length(parm), 2L), 
              dimnames = list(parm, pct))
  ses <- sqrt(diag(vcovHC(object, type=HC_type)))[parm]
  ci[] <- cf[parm] + ses %o% fac
  ci
}

Googling this

I've tried to find the answer by Googling but the results have been slim, below is a short summary of the most interesting articles that I've found.

The sandwich package's vignette has some pieces that I'm not sure exactly how to interpret, all the examples use z-tests but I'm not sure to why and they have this text i section 5:

Predominantly, sandwich estimators are used for inference, such as partial t or z
tests of regression coefficients or restriction testing in nested regression models

From Haan and Levin's "A Practitioner’s Guide to Robust Covariance Matrix Estimation" indicates that this is known at least for the HAC although it didn't give me that much more to work with:

However, to the extent that the estimated HAC covariance matrix exhibits substantial mean-squared error (MSE) in finite samples, the resulting inferences may be severely distorted. For example, substantial variation in the estimated standard error generally causes a t-statistic to take large values (in absolute terms) more frequently than predicted by the limiting standard normal distribution, thereby leading to a tendency to over-reject the null hypothesis in a two-sided test.

Going to the original 2011 paper from Cribari-Neto et al where they propose the HC4m alternative they use the quasi-t distribution. The paper is unfortunately beyond my scope of comprehension but it indicates that the t-distribution might not be the way to go:

Under the null hypothesis, it does not follow a Student t distribution (hence the name ‘quasi-t’), but it does have a limiting standard normal distribution

Unfortunately I'm a little lost by their $\tau$ definition:

$$\tau=\frac{c'\hat{\beta}-\eta}{\hat{var(c'\beta)}}$$

In the article they explain that $\eta$ is a given scalar, and $c'$ is a p-vector, unfortunately this does not aid my understanding of what the formula is doing... Although the hypothesis test seems familiar: $|\tau|>z_{1-\alpha/2}$

$\endgroup$
2
  • $\begingroup$ HAC covariance matrices are typically derived asymptotically. Since the t distribution is meant for use in (small) finite samples, the normal is typically used with HAC standard errors. $\endgroup$
    – Charlie
    Commented Sep 20, 2013 at 21:49
  • $\begingroup$ @Charlie thanks, I found "an answer" that will have to do for now, see below $\endgroup$
    – Max Gordon
    Commented Sep 27, 2013 at 12:41

1 Answer 1

3
$\begingroup$

Prof. Cribari-Neto was kind enough to suggest an article for this topic:

F. Cribari-Neto and M. da G. A. Lima, “Heteroskedasticity-consistent interval estimators,” Journal of Statistical Computation and Simulation, vol. 79, no. 6, pp. 787–803, 2009.

In this article he uses the normal distribution for the confidence intervals. As I wrote in my question this is also supported by my resampling study on the subject where the t-distribution gives considerable over-coverage (96-97 %) for samples $\leq 50$.

A quick review of the t-distribution

Unfortunately I’m not certain to exactly certain to why this is. From what I understand Gosset’s student t-distribution is based upon the uncertainty of the $S$ estimation of $\sigma$. A vital part to the t-distribution are these two formulas:

$$Z = (\bar{X}_n-\mu)\frac{\sqrt{n}}{\sigma}$$

$$V = (n-1)\frac{S^2_n}{\sigma^2}$$

Where he noted that the $V$ follows the $\chi^2$-distribution. Now if we combine these two to get the t-value:

$$T = \frac{Z}{\sqrt{V/v}} = \frac{(\bar{X}_n-\mu)\frac{\sqrt{n}}{\sigma}}{\sqrt{(n-1)\frac{S^2_n}{\sigma^2}/v}} = \frac{(\bar{X}_n-\mu)\frac{\sqrt{n}}{\sigma}}{\frac{s_n}{\sigma\sqrt{v}}\sqrt{(n-1)}}=\frac{(\bar{X}_n-\mu)\sqrt{n}\sqrt{v}}{s_n\sqrt{(n-1)}}$$

By working out that $ν = n − 1$ it further simplifies to:

$$T = \frac{(\bar{X}_n-\mu)\sqrt{n}}{s_n}$$

I guess the $V$ estimation is changed in this case, if the uncertainty of the $S_n$ is less then we're left with the $Z$, in formula language:

$$V = \lim_{S_n \to \sigma} (n-1)\frac{S^2_n}{\sigma^2} = (n-1)$$

Hmm... I hope that I got the formulas right, Wikipedia often has minimized the simplification sections and I thought it would be nice to have it here just to follow what exactly is happening (for us not working with maths on an every-day basis I find the connection between the raw form and the simplified a little hard to digest).

It would be great if someone here could explain exactly why this happening with the HC-estimate.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.