# Inconsistent outcomes of boostraped hypothesis tests on max and median

I am trying to make an hypothesis test using bootstrapping.

I compute a quantity Q from a sample set (the exact calculation should not be relevant, but let's say that Q is the average time between two particular events). I have N different sample sets, so that I have Q1...QN.

I am interested on the following two quantities:

• Q_max: this is the maximum of (Q1, ... QN)
• Q_05: this is the median of (Q1, QN)

Q_max and Q_05 are therefore two different statistics which depend on N different sample sets.

I want to test the hypothesis that Q_max is larger than 0, and also that Q_05 is larger than 0, i.e., the maximum average and the median average are larger than 0. Note: the same procedure should work for every percentile.

I start with the null hypothesis which assumes that all the Qi are 0 (which would imply that the median and the maximum and all other percentiles are also 0), i.e., I subtract the mean mu1 from the first sample s1 to obtain sample p1, so that Q1 is zero, etc.

Then I apply bootstrap as follows. I re-sample (with replacement) from p1 and compute the average E1, I do the same for all samples so that I get all averages E1..EN. Then I compute E_max=max(E1..EN) and E_05=median(E1..EN). I do the same process 10000 times, now I have an empirical distribution of E_max and E_05.

Now I can compute the p-value of (the initial) Q_max and Q_05 to reject the null hypothesis. Assume I decide that the max p-value is 0.1.

I have two questions:

• I obtain that the p-value of Q_max is 0.15 (so I cannot reject the hypothesis that the maximum of the means is 0) but I obtain a p-value of 0.06 for Q_05. This implies that the median is larger than 0 but the maximum is not significantly larger. How is this possible? By logical arguments the maximum should also be larger than 0 if the median is larger than 0?
• Is the method described above OK?
• You say "the exact calculation should not be relevant...", but you actually must verify that the distribution of this test statistical is pivotal under the null hypothesis in order to apply the bootstrap for inference. Nov 6, 2014 at 20:12
• Be very wary of trying to bootstrap a maximum. It typically doesn't work. [Even the median can run into difficulties, especially with small samples -- sometimes people modify their bootstrap procedure (such as moving to a smoothed bootstrap) when dealing with the median.] Nov 7, 2014 at 2:00
• @AdamO: yes, we can assume that the distribution is pivotal, I just tried to avoid details which I thought are not relevant Nov 8, 2014 at 17:28
• @Glen_b: what's about testing that the median is >0 on a very large sample and then infer that all quantiles >0.5 (including the max therefore) are also positive? Nov 8, 2014 at 17:35
• I'm not sure what you're getting at - how would that relate to discreteness problems in small samples? Nov 8, 2014 at 22:37

Moreover, trying to test the max of a distribution doesn't make much logical sense (as you note). If you believe the median is >0, there is no need to try to test the max also. In addition, most distributions that people work with are treated as unbounded, meaning that the true population max is $+\infty$. It may help you to read some of CV's treads that pertain to the max. Clicking on the tag () and sorting by votes will let you start with the highest rated threads.
• What kind of data do you have? The max is presumably $+\infty$ (ie >0). Nov 8, 2014 at 17:36