Fushiki (2005) states that

[...] a nonparametric bootstrap sample $x^{* N}=\left\{x_{1}^{*}, \ldots, x_{N}^{*}\right\}$ [is] independently obtained from the empirical distribution \begin{equation} \hat{p}(x)=\frac{1}{N} \sum_{i=1}^{N}\delta\left(x-x_{i}\right) \end{equation}

I understand how bootstrap works: We sample with replacement from $\left\{x_{1}, \ldots, x_{N}\right\}$. With that in mind, I'm not sure if I understand the mentioned equation. Could you please explain the equation?


1 Answer 1


The equation says that the distribution is an average of point distributions (Dirac delta functions). For univariate $x$ it's probably easier to think of the CDF and then go from there.

The CDF is

$$\mathbb{F}_n(x) =\Pr(X\leq x) = \frac{1}{n}\sum_{i=1}^N I(X_i\leq x)$$

This is a step function. It takes a step of $1/N$ at each observation, meaning that there's a $1/N$ probability for each observation.

It would be nice if you could have a density function rather than a CDF, but the CDF isn't differentiable, so you can't. What you can have instead is what's in the equation. If you naively tried to differentiate the CDF with respect to $x$, you'd get zero at most places and infinite spikes at the observations. The $\delta$ notation is a way to say something like that, but rigorously.

It says that you get the bootstrap distribution by averaging $N$ other distributions. Each of these other distributions has 100% of its probability at one point: $\delta(x-x_1)$ is notation for "a distribution with 100% probability on $x_1$". So, the notation says that the way you pick a number from the bootstrap distribution is to pick one of the $N$ observations, and then choose that number.

The reason this isn't just pointless games with math notation is that it works in more complicated cases. If $x$ is a point on the surface of the Earth, you might have to think for quite a while to decide how to define the CDF, but it's easy to say the way you pick a location from the bootstrap distribution is to pick one of the $N$ observations, and then choose that location.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.