# Selecting correct priors for Bayesian A/B Testing

I'm running an A/B test. The test is a funnel that, at some step in the funnel, sends half the population to experience A, the other half to experience B. Traditionally everyone saw experience A at this step in the funnel, so I have plenty of historical data for my prior on A. However, I have no data at this step in the funnel for Group B. I do, however, have data on Group B, just at a different step in the funnel.

I expect both groups to convert at similar rates, but the point of the test is to determine which converts higher.

I can argue for using either historical A data as my prior for B, and at the same time argue for using historical B data (which happens at a different step, but behaves similarly as A).

Conceptually, how would I reconcile between these two choices? Other than using my domain knowledge, is there a more scientific way to select the prior?

And yes, I understand that selecting a prior is inherently subjective, but I'd like to minimize that if possible.

• Why not use the combined data from both A an B? Use the historical data from group A from some particular step $c$ and historical data from group B from some other step $d$. – missingdataguy Jun 4 '15 at 18:20
• To be sure, that makes sense - however is that approach preferable mathematically speaking to, say, using only data from A or B? This approach kind of "splits the difference" by combining data from each. – ilanman Jun 8 '15 at 16:25
• @ilanman it might not be better, but I don't see up front why it would be worse – shadowtalker Jun 13 '15 at 22:55