# Empirical distribution in bootstrap method

Fushiki (2005) states that

[...] a nonparametric bootstrap sample $$x^{* N}=\left\{x_{1}^{*}, \ldots, x_{N}^{*}\right\}$$ [is] independently obtained from the empirical distribution $$$$\hat{p}(x)=\frac{1}{N} \sum_{i=1}^{N}\delta\left(x-x_{i}\right)$$$$

I understand how bootstrap works: We sample with replacement from $$\left\{x_{1}, \ldots, x_{N}\right\}$$. With that in mind, I'm not sure if I understand the mentioned equation. Could you please explain the equation?

The equation says that the distribution is an average of point distributions (Dirac delta functions). For univariate $$x$$ it's probably easier to think of the CDF and then go from there.

The CDF is

$$\mathbb{F}_n(x) =\Pr(X\leq x) = \frac{1}{n}\sum_{i=1}^N I(X_i\leq x)$$

This is a step function. It takes a step of $$1/N$$ at each observation, meaning that there's a $$1/N$$ probability for each observation.

It would be nice if you could have a density function rather than a CDF, but the CDF isn't differentiable, so you can't. What you can have instead is what's in the equation. If you naively tried to differentiate the CDF with respect to $$x$$, you'd get zero at most places and infinite spikes at the observations. The $$\delta$$ notation is a way to say something like that, but rigorously.

It says that you get the bootstrap distribution by averaging $$N$$ other distributions. Each of these other distributions has 100% of its probability at one point: $$\delta(x-x_1)$$ is notation for "a distribution with 100% probability on $$x_1$$". So, the notation says that the way you pick a number from the bootstrap distribution is to pick one of the $$N$$ observations, and then choose that number.

The reason this isn't just pointless games with math notation is that it works in more complicated cases. If $$x$$ is a point on the surface of the Earth, you might have to think for quite a while to decide how to define the CDF, but it's easy to say the way you pick a location from the bootstrap distribution is to pick one of the $$N$$ observations, and then choose that location.