I'm creating an AB testing framework using Bayesian methods. It's a conversion based test, so users land on the site, randomly get assigned one of two experiences (i.e. group A or group B) and then potentially convert. If I run this test, say, every hour, I'll get a number of people who land and convert within that hour. Then I can easily compare which of the two groups converted at higher rates.
Some people, however, may take 2 hours to convert. Some may take 2 days. I want my model to take into account the fact that one group may convert at a longer time after landing, than the other group.
Does anyone know of a smart way to account for the time-to-conversion component? I'm thinking of comparing conversion rates among cohorts rather than based on time, but after googling around for how people have approached this type of problem, I haven't read anything about it. Surely people aren't ignoring this aspect of their tests.