# Is the following algorithm for bootstrapping test valid?

I have the data (samples) that represent population sizes of different bacterial cultures from 4 distributions for which I would like to compare the ratio of the means between them. Specifically:

$$Ratio_{1} = \frac{\overline{X_{1}}}{\overline{Y_{1}}}$$

$$Ratio_{2} = \frac{\overline{X_{2}}}{\overline{Y_{2}}}$$

What I'm interested into is to check whether $$Ratio_{1} = Ratio_{2}$$ or not.

I've implemented a bootstrap test based on the data, but I'm wondering whether the algorithm I'm using is appropriate.

For example, if I have the following data:

$$X_{1} = (10000, 9000, 9000, 7000, 14000, 16000, 13000, 11000)$$ $$Y_{1} = (50000, 80000, 30000, 60000, 30000, 15000, 70000, 40000)$$ $$X_{2} = (20000, 45000, 31000, 50000, 25000, 39000, 21000, 42000)$$ $$Y_{2} = (400000, 380000, 490000, 200000, 330000, 220000, 340000, 560000)$$

where numbers within the tuples represent the observed population sizes.

I create four new data sets whose values are $$X_i' = x_i - \overline{X_i} + \overline{Z_{X12}}$$ and $$Y_i' = y_i - \overline{Y_i} + \overline{Z_{Y12}}$$ where $$\overline{Z_{X12}}$$ and $$\overline{Z_{Y12}}$$ are the means of the combined sample.

Then I resample several 1000 times and calculate the D statistic based on this

$$D = \frac{\overline{X_{1}}}{\overline{Y_{1}}} - \frac{\overline{X_{2}}}{\overline{Y_{2}}}$$

The two-sided p-value of the test is then calculated as the proportion of boostrap samples where the absolute difference $$|D|$$ is greater than or equal to $$|D_{obs}|$$.

Is this the correct way to do it or is there something wrong? Also is the p-value calculated in the right way?

• Are you actually interested in the specific pairing $(X_{11},Y_{11}),(X_{12},Y_{12}),(X_{13},Y_{13}),\ldots$, or is this about the ratio between any pair of values in $X_1$ and $Y_1$ (same for $X_2$ and $Y_2$)? Commented Jan 24, 2023 at 22:38
• Also note that "Ratio1=Ratio2" is not a valid null hypothesis, because these are defined to be the values actually computed from the data if I interpret the notation correctly. You can just compute $\bar X_1, \bar X_2, \bar Y_1, \bar Y_2$, and Ratio1 wil be equal to Ratio2 or not. No test needed for this. I suspect you want to test $\frac{\mu_{X1}}{\mu_{Y1}}=\frac{\mu_{X2}}{\mu_{Y2}}$, where the $\mu$ are the underlying unobserved means. Alternatively you may want to test $E\frac{X_1}{Y_1}= E\frac{X_2}{Y_2}$, if you want to refer to "fixed pairs" of observations. Commented Jan 24, 2023 at 22:44
• You should not combine the samples; there are no occurrences of $Y_2$ that are less than the largest occurrence of $Y_1$, and similarly for $X_2$ and $X_1$, and that's a clear indication that the $X$ values are not homogeneous, nor are the $Y$ values. Commented Jan 24, 2023 at 22:46
• It would be important to clarify your hypothesis; note that hypotheses are about population quantities (generally population parameters), not sample statistics. What are you actually trying to find out here? (not in terms of sample quantities; what's the research question that this is supposed to be informative about?) Commented Jan 24, 2023 at 22:47
• @ChristianHennig the values Xi and Yi are not paired so I can't really look at that. And you're right, I'm interested in the ratios of the underlying unobserved means. Commented Jan 25, 2023 at 1:09

The most straightforward way to bootstrap your statistic is to repeatedly sample from each of the four subsamples, constructing a bootstrap test statistic at each step, and compare the results to 0 (the null hypothesis):

X1 <- c(10000,9000,9000,7000,14000,16000,13000,11000)
Y1 <- c(50000,80000,30000,60000,30000,15000,70000,40000)
X2 <- c(20000,45000,31000,50000,25000,39000,21000,42000)
Y2 <- c(400000,380000,490000,200000,330000,220000,340000,560000)

D <- mean(X1)/mean(Y1) - mean(X2)/mean(Y2)

boot_D <- rep(0, 10000)
for (i in seq_along(boot_D)) {
boot_D[i] <- mean(sample(X1, replace=TRUE)) / mean(sample(Y1, replace=TRUE)) -
mean(sample(X2, replace=TRUE)) / mean(sample(Y2, replace=TRUE))
}


with result:

> mean(boot_D < 0)
[1] 0


This strongly indicates that the ratios of the means are not the same.

Combining the $$1$$ and $$2$$ sub-populations is fine when they are drawn from the same underlying population, but not otherwise; here, the evidence strongly indicates that this is not the case:

• I believe this approach would depend on whether the quantity is pivotal, otherwise, there's a bias with choosing 0 as the critical/threshold value. Commented Jan 25, 2023 at 16:54
• Could you please explain also this "Combining the 1 and 2 sub-populations is fine when they are drawn from the same underlying population, but not otherwise;" or at least point me to some article/textbook where the theory behind is explained? For example here en.wikipedia.org/wiki/… they combine the samples (to create a new dataset they use means of both samples, which is what I was doing) even if they are from different populations (though that test compares means, not ratios) or do I understand it wrong? Commented Jan 25, 2023 at 20:56
• In the Wikipedia article, they are comparing the means of two samples, and, under the null, they have the same means, so we can combine the samples. In this case, we are not comparing the actual values of sample 1 and sample 2, merely the ratio between the $X$ and $Y$ elements of both. Consequently, the null makes no assumptions about the means of $X1$ and $X2$ and $Y1$ and $Y2$ being the same, and the plot makes it clear that they aren't. If we combine them, we are contaminating our test with $X2/Y1$ and $Y2/X1$, which is not what we want to test (brief because in a comment.) Commented Jan 25, 2023 at 21:11
• @jbowman Right, that makes sense. Thank you! Commented Jan 26, 2023 at 11:41