You serve two different versions of a website to customers, with the aim of seeing which, if either, is overall better. The two website versions are alternated by time of request arriving on the web server, so first would be one version, then for a new user (different IP address) the alternate version and so on.
Because the users are not really matched, and it is unclear if normality holds, the Wilcoxon–Mann–Whitney test seems a good choice. A test could be done, of those customers who bought something where the value in dollars is the random variable. For those customers who do not purchase, a test could be done with the time (delta between first and last request) spent on the website the random variable. Both tests would be one-sided as one of the website versions (the newer one is designed to be better and these test would be to measure if this is the case).
How would the required sample size to be used calculated? Wikipedia [https://en.wikipedia.org/wiki/Mann–Whitney_U_test] says “[i]t is a widely recommended practice for scientists to report an effect size for an inferential test”. But for calculating the sample size you need effect size as an input? Is sample size orthogonal of what test you are using, or the test you are using an input to the sample size calculation? How is an effect size a priori specified (although problem dependent, a simple example would be good). Because there are two tests, does this impact the sample size calculation?
A simulated answer to help understand this may be a good way of helping understand the issues here.