Bootstrap confidence interval on heavy tailed distribution

Question

I read from Wikipedia:

... if one performs a naive bootstrap on the sample mean when the underlying population lacks a finite variance (for example, a power law distribution), then the bootstrap distribution will not converge to the same limit as the sample mean. As a result, confidence intervals on the basis of a Monte Carlo simulation of the bootstrap could be misleading. Athreya states that "Unless one is reasonably sure that the underlying distribution is not heavy tailed, one should hesitate to use the naive bootstrap".

Here's what I understand:

a) Distributions with heavy tails may have infinite variance, or mean (Ex: cauchy distribution)

b) Heavy tailed means that there are a few outliers that are very different from the most of the samples. And these outliers have non-negligible impact on the future statistic procedures.

c) Log-normal or exponential distributions have heavy tails

Here's the question:

1. Are my understandings (a, b, c) right?
2. Can Bootstrapping be used to estimate confidence interval of mean or variance of lognormal, or exponential population?
3. Why does Bootstrapping fail in case of heavy tail?

Please do not use statistical equations (ex: expectations...) as I do not have the background to understand them.

Answer 1

a) Distributions with heavy tails may have infinite variance, or mean (Ex: Cauchy distribution)

True.

b) Heavy tailed means that there are a few outliers that are very different from the most of the samples. And these outliers have non-negligible impact on the future statistic procedures.

Partly true. This might be how it looks in a realization drawn from the distribution, but the outliers are not part of a discrete/separable component (the distribution is typically unimodal, meaning the tails decay gradually - just really slowly)

c) Log-normal or exponential distributions have heavy tails

Partly true. (updated with info from @glen_b's comment) These distributions are both heavier-tailed than the Gaussian distribution, but the exponential is not heavy-tailed enough to cause difficulties. The log-Normal has a finite variance, so is theoretically OK, but can cause problems. Pareto and Cauchy (and other extreme t distributions, e.g. Student $t$ with 2 df) are in the "highly problematic" category.

Answer 2

Here's what I understand: a) Distributions with heavy tails may have infinite variance, or mean (Ex: cauchy distribution) b) Heavy tailed means that there are a few outliers that are very different from the most of the samples. And these outliers have non-negligible impact on the future statistic procedures. c) Log-normal or exponential distributions have heavy tails

Here's the question: Are my understandings (a, b, c) right? Can Bootstrapping be used to estimate confidence interval of mean or variance of lognormal, or exponential population? Why does Bootstrapping fail in case of heavy tail?

1)a) They can have an undefined mean or variance. Under some specifications, that is represented as infinite. 1)b)It depends upon the procedure being used. If you have a Cauchy distribution and you are using the sample mean to estimate the center of location then the answer is yes. If you have a Cauchy distribution and a large enough sample and are using a Bayesian method or Rothenberg's estimator then the answer is 'no.' It could but it need not. 1)c)Most definitions of heavy tails are those greater than the exponential distribution so the exponential is not a heavy-tailed distribution see

Bryson, M. (1974). Heavy Tailed Distributions: Properties and Tests. Technometrics 16(1):61-68 (February 1974).

2) Yes, bootstrapping can be used for either. However, it somewhat begs the question as to why you would use it for either. If you really believed those were the distributions, then there are good parametric tools for both.

3) Yes, sort of. An example of where resampling is used would be Theil's regression across two Cauchy variables. There is a special limited sampling case for Theil's regression that is basically a bootstrap. The issue is that you are not seeking a mean as none exists. You would be seeking the median of the joint set.

There is nothing intrinsic to bootstrapping that prohibits its use with heavy tails but you cannot use it to find something that does not exist, such as a variance or a mean. As with any problem where you have fewer good properties, the usefulness of bootstrap will be greatly reduced.

Answer 3

Bootstrapping the sampling distribution of a sample mean will work (in the sense of being consistent as n diverges) only if a Central Limit Theorem applies, thus existence of the variance is practically required. See Mammen, 'When Will Bootstrap Work, Springer 1992. The intuition is indeed that otherwise observations remain influential. In most counterexamples, the bootstrap distribution fails to converge anywhere.