# What is a "bootstrap from the Empirical CDF"?

I am given the following set of questions.

In all the questions the basic background is a sample X_1, ... , X_n.

1. If n=3 and you observe the numbers 1, 2, and 4, what is the sample median.
2. If you draw a bootstrap sample (of size 3 with replacement) from the empirical cdf for the data in 1, what are the possible values of the median of these bootstrap samples.
3. What are the bootstrap probabilities of the values you named in 2?
4. How many essentially different bootstrap samples are there?

The answer to question 1 should be 2, because that's the middle value of $$\{ 1, 2, 4 \}$$.

For question 2, I don't know what it means to "draw a bootstrap sample... from the empirical cdf". As far as I know, the empirical cdf would take the form

\begin{align*} \hat F_3(x) = \begin{cases} 0, \text{ for } x < 1\\ 1/3, \text{ for } 1 \leq x < 2\\ 2/3, \text{ for } 2 \leq x < 4\\ 1, \text{ for } 4 \leq x. \end{cases} \end{align*}

I know how to take a bootstrap sample of size 3 with replacement from $$\{ 1, 2, 4\}$$. You just take a random sample of size 3 from the set, so you might get a result like $$\{ 1, 2, 2 \}$$ for example. But how do you take a bootstrap sample from the empirical cdf?

Edit: I will proceed assuming that I am in fact supposed to sample from $$\{ 1, 2, 4 \}$$, and see if I can come up with sensible answers to questions 2, 3, and 4.

For question 2, the possible medians are $$1, 2$$ and $$4$$, as can be seen from the possibility of the bootstrap samples being $$\{ 1, 1, 1 \}$$, $$\{ 2, 2, 2 \}$$, or $$\{ 4, 4, 4 \}$$.

For question 3, by "bootstrap probabilities" I assume I am being asked for the probability of selecting $$1, 2$$ or $$4$$ in the bootstrap sample. I figure the answer in all cases should be $$1/3$$, because we're taking a random sample from a population of size $$3$$.

For question 4, we are asking for the number of selections from $$3$$ elements with replacement when order doesn't matter, and I this is equal to $$\binom{5}{3} = 10$$, which is confirmed fairly quickly manually.

Edit 2: Following Glen_b's advice below.

The possible resamples should be

1. 1, 1, 1
2. 2, 2, 2
3. 4, 4, 4
4. 1, 1, 2
5. 1, 1, 4
6. 2, 2, 1
7. 2, 2, 4
8. 4, 4, 1
9. 4, 4, 2
10. 1, 2, 4

We have a median of 1 in three samples, so the probability of 1 being the median in a bootstrap sample of size 3 is 3/10. The probability for 2 is 4/10, and for 4 it's 3/10. This answers question 3.

For question 4, there are 10 essentially different bootstrap samples, which can be found either combinatorially or through manual enumeration as above.

Edit 3: Following EdM's advice, I wrote out all 27 possible bootstrap samples and counted 7 having median 1, 13 having median 2, and 7 having median 4. So I guess the probabilities are 7/27, 13/27 and 7/27 respectively.

I think I was right that there are 10 "essentially different" bootstrap samples, but clearly some information was lost when I passed from considering all bootstrap samples to only considering those unique up to order.

• 1. See stats.stackexchange.com/help/on-topic under the paragraph that begins with the word Homework. 2. Question 3 is explicitly asking about the probabilities of the medians being 1, 2 and 4 (the overall proportion of bootstrap samples that yield those medians -- you can compute these exactly since there's a small number of possible resamples; I suggest you list out all the possible resamples to make sure you understand what you're calculating). Sep 13 at 5:28
• @Glen_b I've edited my post and attempted to follow your advice. One of the main points of confusion regarding this collection of questions is the phrase "bootstrap sample from the empirical cdf". Is that meaningful to you? To me it makes no sense, because as far as I know bootstrap samples are taken from an obtained data set, like $\{ x_1 = 1, x_2 = 2, x_3 = 4 \}$. Sep 13 at 18:42
• With respect to Question 3, there are 3*3*3 = 27 possible results of a standard bootstrap sample from {1,2,4}. You seem to have delineated 10 distinguishable samples (ignoring the sampling order), but some of those 10 are more likely than others so your proposed answer might not be correct. With this type of situation it's safest to start systematically, get all the possibilities (27 here), and work from there. "Sample from the empirical cdf" is just terminology for sampling from the data set, distinguished from, say, sampling from a parametric distribution meant to model the data set.
– EdM
Sep 13 at 18:57
• The distinction being made is that sampling from the ECDF admits a unique valid interpretation (namely, it's exactly like sampling from any distribution), whereas "sample from the dataset" can legitimately be viewed in at least two distinct ways: with and without replacement.
– whuber
Sep 13 at 19:17
• That's why doing practice problems is so important. If you think that you've answered the problem now, why don't you post an answer yourself? That's OK on this site.
– EdM
Sep 14 at 18:58

"Sample from the empirical CDF" appears to be lingo meaning "sample with replacement from the dataset." Therefore, in question 2, it becomes clear that the questioner is asking what the median could be of samples taken with replacement from $$\{1, 2, 4\}$$.
For question 3, I think I'm being asked for the probability of selecting a sample (with replacement) from $$\{ 1, 2, 4\}$$ such that the median of that sample is 1, 2, or 4. The most direct way to deal with this is to enumerate all the samples, which all have equal probability of being chosen, and count up the number of samples that have a given median.