How can I determine statistical significance in an A/B test in which the KPI is dependent upon two variables - one bernouli and one continuous? In my work in online marketing, we frequently run A/B or multivariate web page tests. The Key Performance Indicator for these tests is overall Revenue. The treatment that nets the most revenue, either by influencing more conversions to sale, or by influencing a user to spend more per transaction, or a combination of these two, is the winner. 
However, how can we determine we have a statistically valid Revenue impact in a particular test? The standard-issue significance test does is not useful here, since the Revenue impact is obviously due to a combination of probability to convert + dollar value of conversion.
We have played with a version of Overall Evaluation Criterion from the Taguchi playbook, but honestly I'm not statistically sophisticated enough to know if this approach is valid. 
I appreciate any suggestions. 
 A: You might consider a simple Bayesian model of the two treatments: Let whether each visitor makes a purchase under each treatment be Bernoulli random variables with respective parameters, and let purchase amounts be lognormal random variables with respective parameters log-scale parameters and equal shape parameters. (Lognormal is here used for illustration, taking for granted that it's a reasonable fit.)
Having done that, we can build a posterior predictive model for the average sale of each treatment. (See explanation in code comments.)
import pymc as pm, numpy as np
from matplotlib import pyplot as plt

# Priors for probability of a purchase, and for the log-scale parameters
# The beta distribution is conjugate to the Bernoulli and binomial, so it's a commonly used prior
thetaA = pm.Beta("pA", .5, .5)
thetaB = pm.Beta("pB", .5, .5)
muA = pm.Normal("muA", 0, 10e-6)
muB = pm.Normal("muB", 0, 10e-6)

# Now let's make some toy data
# Note that treatment b has a lower probability of purchase, but a higher average purchase
# These are the true purchase probabilities
observedPurchaseA = pm.rbernoulli(0.075, 1000)
observedPurchaseB = pm.rbernoulli(0.03, 1000)
obsPurchaseAmountsA = np.multiply(observedPurchaseA, np.random.lognormal(2, 0.25, 1000)) 
obsPurchaseAmountsB = np.multiply(observedPurchaseB, np.random.lognormal(2.1, 0.25, 1000))

obsPurTrA = pm.Bernoulli("purchasesA", thetaA, value=observedPurchaseA, observed = True)
obsPurTrB = pm.Bernoulli("purchasesB", thetaB, value=observedPurchaseB, observed = True)
obsAmtTrA = pm.Lognormal("amtA", muA, 1/0.25, value = obsPurchaseAmountsA[obsPurchaseAmountsA > 0], observed = True)
obsAmtTrB = pm.Lognormal("amtB", muB, 1/0.25, value = obsPurchaseAmountsB[obsPurchaseAmountsB > 0], observed = True)

# Since we observed more purchases in one treatment than the other, we have to contend with unequal sample sizes for purchase amounts
# That's not a problem in Bayesian analysis, but more data generally means a narrower posterior distribution 
# In other words, we'll be surer of the parameter for which we've observed more purcahses

# What we're really interested in is the posterior predictive distribution: The distribution of future purchases based on the data we've seen
# Rather than compute that directly, here I'm using simulation to compute the average sale per visitor at each step in the MCMC chain
@pm.deterministic
def expectedA(mu = muA, theta = thetaA):
    return np.mean(np.multiply(pm.rbernoulli(theta, 1000), np.random.lognormal(mu, 0.25, 1000)))

@pm.deterministic
def expectedB(mu = muB, theta = thetaB):
    return np.mean(np.multiply(pm.rbernoulli(theta, 1000), np.random.lognormal(mu, 0.25, 1000)))

model = [thetaA, thetaB, muA, muB, observedPurchaseA, observedPurchaseB, obsPurchaseAmountsA, obsPurchaseAmountsB, obsPurTrA, obsPurTrB, obsAmtTrA, obsAmtTrB, expectedA, expectedB]#, purAsim, purBsim, amtAsim, amtBsim]
mcmc = pm.MCMC(model)
mcmc.sample(30000,10000)

# Now let's compare the distribution of average sales
ax = plt.subplot(211)
plt.xlim(0, np.max(np.concatenate((mcmc.trace('expectedA')[:],mcmc.trace('expectedB')[:]))))
plt.hist(mcmc.trace('expectedA')[:], bins = 25)
ax = plt.subplot(212)
plt.xlim(0, np.max(np.concatenate((mcmc.trace('expectedA')[:],mcmc.trace('expectedB')[:]))))
plt.hist(mcmc.trace('expectedB')[:], bins = 25)

Now a look at those posterior plots:

The first treatment clearly outperforms the second. The formal decision rule for this could rely on comparing the credible intervals for the two treatments. Another option is to compute the distribution of the difference between treatments.
It's also a simplifying assumption to use equal shape parameters, but programming them into the model is straight-forward. But again, the data here is just a toy example; in practice you'd want to use a model with a well-evaluated fit.
