We've run a split test of a new product feature and want to measure if the uplift on revenue is significant. Our observations are definitely not normally distributed (most of our users don't spend, and within those that do, it is heavily skewed towards lots of small spenders and a few very big spenders), so we've decided on using bootstrapping to compare the means, to get round the issue of the data not being normally distributed.
So my results show that we do have an uplift of around 8% vs. control. I now want to calculate how confident I can be in this uplift. Is it as simple as measuring the proportion of the probability density function below zero, for the PDF that is test group PDF minus control PDF? (e.g., that portion reflects the % chance that my 2 PDFs are not different?)
Any help would be much appreciated.