Please correct me where I'm wrong:

My understanding of bootstrapping is that it is a way to estimate the distribution of some statistic (mean, standard error, Pearson's correlation coeff, etc), given only one sample. So if I want to estimate the mean of a population using bootstrap methods, I generate many bootstrap samples, compute the mean of each of these bootstrap samples, and then use the distribution of those values to deduce where the unknown population mean is likely to fall and compute a confidence interval for the statistic.

But how are the bootstrap samples generated? There is a scikit bootstrap module and I see that it has a bootstrap method to compute confidence interval for a given statistic: see first function, def(ci).

The first estimator is the empirical distribution function, which should be an array that the statistic of interest can be computed on. How is this empirical data used to generate the bootstrap samples?

To extend this question, if I want to compute a 95% confidence interval for the Pearson correlation coefficient between two random variables x and y, and I pass data = [(x1,y1), (x2,y2), ... (xi,yi), ... (xn,yn)] to the implementation of bootstrap CI, does that mean that (x1, ..., xn) and (y1, ..., yn) are generated independently of each other for each bootstrap sample that is generated?

  • $\begingroup$ Not clear how far this is a general question and how far a highly specific question on how to use scikits for the purpose. $\endgroup$ – Nick Cox May 22 '13 at 17:58

The short answer is that - at least in the simple cases - the observations are sampled with replacement. Imagine writing each of the data values on an n sided die and rolling the die n times.

If you're trying to bootstrap a correlation, you resample the data in pairs $(x_i,y_i)$. If you think of your data as two columns, each row is an observation, and you resample the observations (rows).

Here's an example:

enter image description here

More generally, think of a matrix of data where the observations (rows) are resampled.

(This is not a suitable resampling scheme for every situation, though. There are a plethora of bootstrap schemes.)

  • $\begingroup$ I know this is an old question... I'd rather not open a new question to avoid adding entropy. From what I know from bootstrap, your explanation is completely right. If I try to apply it for example to the mean, every time I resample I get a value around the observed mean and indeed at the end I can establish a conf. int. around that mean. However, if I apply it to pearson correlation coefficient, every time I resample I get a value around 0. Which, when I think about it, makes sense: the resampling just breaks the correlation. But how to establish conf. int. then? Thanks! $\endgroup$ – lrnzcig Apr 15 '15 at 9:38
  • $\begingroup$ @lrnzcig My answer above already discusses this - you keep the pairs together. That maintains the correlation close to what it is in the data, rather than losing it. If you break the pairing (fail to keep $x_i$ with $y_i$, but instead allow it to pair with any $y$-value -- $y_j$, say) then you'll lose the correlation. $\endgroup$ – Glen_b Apr 15 '15 at 10:46
  • $\begingroup$ @lrnczig -- I've added an example to illustrate that the resampling I described doesn't break the correlation. $\endgroup$ – Glen_b Apr 15 '15 at 11:22

The bootstrap is one of a plethora of estimation techniques based on the empirical distribution function of the data, $x$:

$$\mathbb{F}(t) = \int_{0}^t \frac{\sum_{i=1}^n I(s > x_i)}{n} ds$$

In the multivariate setting, you consider rows of observations perfectly correlated when bootstrapping. This prevents us from sampling post menopausal males in cancer risk studies.

With a sample cumulative distribution function but you can draw samples from it based on any random sampling technique, which is a de facto tool in almost any statistical package. Drawing samples from this is equivalent to just assigning $1/n$ probability to every jointly observed row in your data. This means that, in your case, $(x_i, y_i)$ pairs would have to be sampled jointly. Permutation testing on the other hand allows you to randomly rearrange the columns of jointly observed rows of data and perform resampling tests based on those values.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.