I have goal of measuring CTRs of several titles of an article on a website using Bayesian approach.
In a simple setup, what one will do is to select Beta Prior (for example Uniform distribution) and run series of experiments until he gets credible intervals narrow enough to select proper title.
Let's assume now that I run the experiment on several websites simultaneously and want to understand statistics regarding each title-website pair - what should I do in this case?
Again, the simplest way is just to treat all pairs as independent and reuse approach from first phase.
What is however the best way to use the shared information, like number of trials and successes of particular title on all websites, while doing the experiment? It seems that this can lead to the credible results much faster (i.e. maybe one site is just not converting at all as there only bots there, or one title is performing much better on all websites)
What is the best way to update posterior distributions (or to reselect priors) using shared information between the entities?
site_id title_id impressions clicks 123 t1 10051 150 123 t2 560 1 . . . 789 t1 101 15 789 t2 1050 2 .