3
$\begingroup$

Suppose I am willing to assume a particular likelihood function for an applied statistics problem. I am able to derive the MLE for the parameter $\theta$, which I will call $\hat{\theta}$. I can also derive a standard error for $\hat{\theta}$.

I am also interested in generating a point estimate and confidence interval for some transformed parameter $\tau = g(\theta)$. A statistics textbook might remind me that $\hat{\tau} = g(\hat{\theta})$ is the MLE for $\tau$ due to the equivariant property of the MLE and that I can use the delta method to get standard errors for $\tau$.

I can also use the parametric bootstrap to get confidence intervals for $\hat{\tau}$ (and, for that matter, $\hat{\theta}$), since I am assuming a parametric model.

I have a suspicion that the parametric bootstrap is a "safer" way to form confidence intervals than the delta method, but that the delta method is emphasized in statistics classes because it is a nifty mathematical result and because of historical reasons (for example, statistical instruction predates fast computers). Downsides to the delta method are:

  • CIs based on a normal approximation often don't work because asymptotic normality hasn't "kicked in" yet
  • It may be hard to simply time consuming to derive
  • Even when it is tractable, it's easy to make math mistakes in deriving the standard error

My question is: in practical applications (rather than when we are doing school or theory), do people really find the delta method more useful than the parametric bootstrap in many cases? Or should I think of the parametric bootstrap as my "default" method for computing CIs in this situation (with the possibility of overruling this default, if there are special circumstances to the problem at hand)? Or perhaps these questions are wrong-headed for some reason I haven't considered?

$\endgroup$
1
  • $\begingroup$ Would (cloud) compute cost ever be of concern in your application? Resampling many times may be fast for one application, but for many parallel applications the compute requirement may build up and some might find the compute bill unwelcoming in a business context. $\endgroup$
    – B.Liu
    Dec 30, 2021 at 18:36

2 Answers 2

3
$\begingroup$

You will often be disappointed in the confidence interval coverage probability in at least one of the two tails when using either the bootstrap or the delta method. This is one area where traditional statistical education, which focuses on frequentist methods, has done us a disservice. Contrast this with Bayesian modeling and using MCMC to generate thousands of samples from the multivariate posterior distribution of all the parameters. Given these samples, one merely transforms the parameters to any desired new parameter, transforming all of the posterior samples into the new metric, then computes things such as highest posterior density uncertainty interval.

The delta method assumes that the sampling distribution of the derived parameter is approximately symmetric, which is very often not the case. The Bayesian solution is not only easy to program and doesn't assume that anything is normal (or even symmetric) but provides exact inference on the derived parameter to within simulation error.

$\endgroup$
2
  • $\begingroup$ Thanks! I would expect the delta method to often yield poor coverage due to the asymptotic-based approximations not working well. But what is the intuition for why the coverage will be bad for the parametric bootstrap? Couldn't I use a percentile interval, which need not be symmetric? A main drawback with the parametric bootstrap (as contrasted with the non-parametric bootstrap) is that I need to assume a likelihood, but I would also need to do this for a Bayesian approach. $\endgroup$
    – frelk
    Jan 5, 2022 at 0:27
  • $\begingroup$ I have less experience with the parametric bootstrap, but I have run simulations for odds ratios in binary logistic models where the basic bootstrap, BCa bootstrap, and nonparametric bootstrap percentile intervals failed very badly. I think the bootstrap's accuracy depends on the degree to which the bootstrap distribution mimics the sampling distribution. In my logistic model examples it didn't mimic it well. Just remember that the bootstrap is an approximate method, unless you use the expensive double bootstrap to self-correct its errors. I'd rather get exact Bayesian inference. $\endgroup$ Jan 5, 2022 at 13:39
2
$\begingroup$

People really do find the delta method more useful in some cases. It is really impossible to tell which is more appropriate without being given a specific context since the context would largely determine which method is "best" for the given situation. I think that you've considered this to at least some extent, because you've listed some downsides to the delta method that you came up with.

I pick up on some skepticism, or sentiment that the delta method may be favored by academics without being pragmatic in real-world applications where we have modern computational statistics. The skepticism is good, but consider your downsides to the delta method. The first only holds when asymptotic normality hasn't "kicked in" as you put it, but it can "kick in" faster than you'd expect in some applications. Mistakes in the derivation are possible, but not a serious concern since you'd typically not be doing anything wildly different from one application to another. And it usually isn't too time-consuming to derive. If you're willing to use a computer for bootstrapping, then you can use a computer for the delta-method... and I guarantee that it will take far less computation than bootstrapping.

Here's a real-world example of the delta-method that is often used in reality: Greenwood's formula. Greenwood's formula uses the delta-method to come up with standard errors for Kaplan-Meier curves, which are used in survival analysis (medical studies, mostly). This is just one very big example off the top of my head that uses the delta method in a major real-world application, and I'm sure there are more.

I think you can answer your own question if you give yourself a context. Just make pros and cons for bootstrapping and the delta-method like you already did with the delta-method, and ask yourself "for my given situation, which method's pros outweigh the cons?"

$\endgroup$
1
  • 1
    $\begingroup$ My question was more about parametric models (where you might use the parametric bootstrap or find the SE of a function of the MLE vector via the delta method). Your example is very interesting for the non-parametric case, though. Out of curiosity, do you know whether CIs from Greenwood's formula are thought to have better / similar coverage properties than CIs from a non-parametric bootstrap of the Kaplan-Meier curve? $\endgroup$
    – frelk
    Jan 5, 2022 at 2:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.