# Bootstrap confidence interval on heavy tailed distribution

... if one performs a naive bootstrap on the sample mean when the underlying population lacks a finite variance (for example, a power law distribution), then the bootstrap distribution will not converge to the same limit as the sample mean. As a result, confidence intervals on the basis of a Monte Carlo simulation of the bootstrap could be misleading. Athreya states that "Unless one is reasonably sure that the underlying distribution is not heavy tailed, one should hesitate to use the naive bootstrap".

Here's what I understand:

a) Distributions with heavy tails may have infinite variance, or mean (Ex: cauchy distribution)

b) Heavy tailed means that there are a few outliers that are very different from the most of the samples. And these outliers have non-negligible impact on the future statistic procedures.

c) Log-normal or exponential distributions have heavy tails

Here's the question:

1. Are my understandings (a, b, c) right?
2. Can Bootstrapping be used to estimate confidence interval of mean or variance of lognormal, or exponential population?
3. Why does Bootstrapping fail in case of heavy tail?

Please do not use statistical equations (ex: expectations...) as I do not have the background to understand them.

• The sort of heavy tailedness that would be of concern certainly doesn't include the exponential. While the lognormal does have finite variance it can sometimes be heavy tailed enough that the population mean will nearly always exceed all of your sample, which can make inference via a bootstrap tricky. The sample sizes required to get a decent idea of the sample mean from a bootstrap even when the moments are all finite can sometimes be extremely large. – Glen_b -Reinstate Monica Sep 9 '19 at 1:51

a) Distributions with heavy tails may have infinite variance, or mean (Ex: Cauchy distribution)

True.

b) Heavy tailed means that there are a few outliers that are very different from the most of the samples. And these outliers have non-negligible impact on the future statistic procedures.

Partly true. This might be how it looks in a realization drawn from the distribution, but the outliers are not part of a discrete/separable component (the distribution is typically unimodal, meaning the tails decay gradually - just really slowly)

c) Log-normal or exponential distributions have heavy tails

Partly true. (updated with info from @glen_b's comment) These distributions are both heavier-tailed than the Gaussian distribution, but the exponential is not heavy-tailed enough to cause difficulties. The log-Normal has a finite variance, so is theoretically OK, but can cause problems. Pareto and Cauchy (and other extreme t distributions, e.g. Student $$t$$ with 2 df) are in the "highly problematic" category.

• +1. An example of how extreme the lognormal can be (even with all moments finite!), is here. The histogram shows the approximate shape of the distribution of not the sum (and hence the mean) of 50000 lognormals, but of its log -- yet it is still pretty strongly right skew. The population mean is at the 0.97725 quantile of the distribution. That's with a $\sigma$ parameter of merely 4. Larger values of $\sigma$ can be far more extreme still. – Glen_b -Reinstate Monica Sep 9 '19 at 2:30
• Even with that $\sigma=4$ example, at a sample size of 30, more than half the time every value in the sample will be below the population mean. Clearly if you're trying to bootstrap an interval for the mean when all your data are below the mean, it's not usually going to work particularly well. – Glen_b -Reinstate Monica Sep 9 '19 at 2:34

Here's what I understand: a) Distributions with heavy tails may have infinite variance, or mean (Ex: cauchy distribution) b) Heavy tailed means that there are a few outliers that are very different from the most of the samples. And these outliers have non-negligible impact on the future statistic procedures. c) Log-normal or exponential distributions have heavy tails

Here's the question: Are my understandings (a, b, c) right? Can Bootstrapping be used to estimate confidence interval of mean or variance of lognormal, or exponential population? Why does Bootstrapping fail in case of heavy tail?

1)a) They can have an undefined mean or variance. Under some specifications, that is represented as infinite. 1)b)It depends upon the procedure being used. If you have a Cauchy distribution and you are using the sample mean to estimate the center of location then the answer is yes. If you have a Cauchy distribution and a large enough sample and are using a Bayesian method or Rothenberg's estimator then the answer is 'no.' It could but it need not. 1)c)Most definitions of heavy tails are those greater than the exponential distribution so the exponential is not a heavy-tailed distribution see

Bryson, M. (1974). Heavy Tailed Distributions: Properties and Tests. Technometrics 16(1):61-68 (February 1974).

2) Yes, bootstrapping can be used for either. However, it somewhat begs the question as to why you would use it for either. If you really believed those were the distributions, then there are good parametric tools for both.

3) Yes, sort of. An example of where resampling is used would be Theil's regression across two Cauchy variables. There is a special limited sampling case for Theil's regression that is basically a bootstrap. The issue is that you are not seeking a mean as none exists. You would be seeking the median of the joint set.

There is nothing intrinsic to bootstrapping that prohibits its use with heavy tails but you cannot use it to find something that does not exist, such as a variance or a mean. As with any problem where you have fewer good properties, the usefulness of bootstrap will be greatly reduced.