mean_diff = np.mean(insured) - np.mean(non_insured)

# Compute mean of combined data set: combined_mean
combined_mean = np.mean(np.concatenate((insured, non_insured)))

# Shift the samples
insured_shifted = insured - np.mean(insured) + combined_mean
non_insured_shifted = non_insured - np.mean(non_insured) + combined_mean

# Get bootstrap replicates of shifted data sets
bs_replicates_insured = draw_bs_reps(insured_shifted, np.mean, 10000)
bs_replicates_non_insured = draw_bs_reps(non_insured_shifted, np.mean, 10000)

# Compute replicates of difference of means: bs_diff_replicates
bs_diff_replicates = bs_replicates_insured - bs_replicates_non_insured

# Compute the p-value
**p = np.sum(bs_diff_replicates >= mean_diff) / len(bs_diff_replicates)**

# Print p-value, p-value = 0.0

I need help understanding what the numerator is in this code is
p = np.sum(bs_diff_replicates >= mean_diff)

How many bootstrap replicates would you need to run to see re-sample difference > your observed difference?

Here is the P-value when I used inferential stats opposed to bootstrap

P-value is: $2.230615115810486e-31$

I was told that this would somehow help me determine how many replicates I would have to run to see a re-sample difference > than my observed differences.

If you don't have an answer, is there any resources that you can recommend. I am more solid on the code but I need more grounding in the theory


1 Answer 1


I cannot really help you in python, but for choosing the number of replications ($B$), you can calculate the ideal bootstrap. This is the number of replicates that it takes to make the standard error converge.

It seems like you are testing for the difference of means here. Back in 1993, Efron and Tibshirani in An introduction to the bootstrap suggested $B=200$ for the standard error. As for confidence intervals, they suggested $B>1000$. Since this is an hypothesis test, I would personally suggest that you use at least $B=1000$.

Remember also that since 1993, computing power has increased significantly and we can therefore choose a much larger value for $B$. Michael Chernick suggests in his book, Bootstrap Methods: A Guide for Practitioners and Researchers written in 2008, that it is more feasible to choose a larger value for $B$, say $10000$ for the day and age we are living in.

Why so many?

Because computers allow us to obtain so many in such a small amount of time.

Chernick also mentions that the number of replications you choose will depend on the specific problem and the desired accuracy - that the question that you can answer :)

Hope this is helpful and makes sense


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.