# Split testing significance with continuous measures

What technique would you use to understand the statical significance of a test where the target outcome is a continuous measure?

Say there are three tests

Control, A and B each with 50k samples and they generate the following revenues of 1054, 750 and 450.

I've done this with discrete measures like 20 people out of 1000 converted but never continuous measures.

• More specific information about the data, the purpose of the "tests," and the kinds of tests is needed. Would you be able to edit this question to provide that necessary amplification?
– whuber
Jul 16, 2014 at 14:59

What is normally advised is the mean revenue per user, $\mu$, and the error on that mean, $\sigma_\mu$. Since your samples are large you can likely use the central limit theorem to get the error on the mean as the standard deviation of your samples divided by the square root of the number of users. If you do not trust this you can use bootstapping to get the error on the mean. $\mu$ and $\sigma_\mu$ for each sample gives you enough to build three Gaussian distributions which you can compare using standard methods (eg t-test).
CAUTION: It would be a mistake to calculate $\mu$ or $\sigma_\mu$ using the number of purchasers instead of the number of users who could have purchased. You need to account for the fact that some people did not contribute to the revenue and that this can be different between your splits.