Bootstrap hypothesis testing with small sample sizes

Question

I have 2 campaigns (a control and a test campaign), the data are like this:

Capaign      # of launches  Revenue/Visits
 Control             1             2.35 
 Test                1             1.97 

I'm trying to do a hypothesis testing on Revenue/Visits metric. But since I don't know it's distribution, I thought It could be a good choice using bootstraping to simulate more data. But I took a look at this document:

http://www.stat-athens.aueb.gr/~karlis/lefkada/boot.pdf

And I don't know exactly how could I simulate other samples from this data.

What do you suggest?

Answer 1

A non-parametric bootstrap does actually not simulate new data. It only resamples with replacement from the existing data, trying for instance to estimate the variability of a certain statistic.

In your case you only have two observations, if I understand correctly. You can hardly use non-parametric bootstrap in this case in any sensible way. There is only four possible subsets of your data set to sample. This will give you a very poor estimate of the distribution of any statistic you're interested in.

Other resampling methods are abound but they'll all suffer from the fact that your data set is extremely small. Thus, estimating the distribution of a statistic will be too optimistic as data will not contain enough information.