I have recently been looking into the Bootstrap, and I was wondering, if I were to have a sample $X=\{x_1,x_2,\dots,x_N\}$ that has $N$ samples, all i.i.d coming from a normal distribution, $N(\mu,\sigma)$.

In the Bootstrap, $B$ resamples, $X_b=\{x_{b1},x_{b2},\dots,x_{bN}\}$, are obtained by resampling uniformly from $X$ (I know that there are variations with bias-corrections and accelerations, but let's keep that aside for now).

So, my question is, how can I relate the variance of $x_1$ to $x_{b1}$? I have a feeling that they are two different random variables now that the uniform resampling with $\tfrac{1}{N}$ selection probability.

Here is how I think of it. $x_i$ comes from a continuous distribution, that is normal. $x_{bi}$, however, comes from a discrete distribution, that is a uniform selection of $x_1, x_2, \dots, x_N$. I think that the probability space of $x_i$ maps somehow into that of $x_{bi}$, but I do not know what sort of a relationship is between them. I am pretty sure it is some form of a bijective mapping $x_i \to x_{bi}$, where $x_{bi}=f(x_i)$, but I cannot think of a way to map "normally distributed numbers to make them uniformly distributed".

My purpose is to simply understand how the resampling process works, how it affects the variance, and whether the variance of $x_{bi}$ can be related to the variance of $x_i$? I am just interested in how it works.

  • $\begingroup$ When you say resample uniformly, do you mean that the rank/database position of the sample was chosen with equal probability for each? Or is the sampling based on the magnitude of X in order to achieve a uniform distribution of values in X? If the former, are the sample positions randomised or structured prior to resampling? $\endgroup$ – ReneBt Mar 29 '18 at 12:33

The question can be rephrased as: what is the distribution of $x_{b_1}$? In other words consider $b_1$ being uniformly distributed in $\{1,2,...,N\}$ and $x_1,x_2...x_N$ being independent and distributed according to $N(\mu,\sigma^2)$, what is the distribution of $x_{b_1}$?

$$P(x_{b_1}\in A)=\sum_{k=1}^N P({b_1}=k)P(x_k\in A|{b_1}=k)$$

Since ${b_1}$ and $X$ are independent (which is natural since ${b_1}$ is generated with a random generator):

$$P(x_{b_1}\in A)=\sum_k P({b_1}=k)P(x_k\in A)=\frac{1}{n}\sum_kP(x_k\in A)=P(x_1\in A)$$ because all $x_k$ have the same distribution. Thus $x_{b_1}\sim N(\mu,\sigma^2)$ and thus has variance $\sigma^2$. In other words, $x_{b_1}$ is the mixture of several times the same distribution thus has this distribution.

Next question: what is the distribution of $X_b$? It's more difficult I think.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.