0
$\begingroup$

I understand the basics of Thomson Sampling, but how is it implemented in practice? If there are three variants each with a 1/3 of traffic allocated to them on day 1, how is traffic dynamically allocated on day 2, 3, and so forth? All of the examples I've read show a simulation of posterior sampling, but how can this be done in a real world application of bandit testing?

$\endgroup$
0
$\begingroup$

I guess that by "in practice" you meant that you want to group your reward feedback and action decision by batch (e.g. every day).

If you do a single Thomson sampling round and allocate 100% of your traffic to the best arm (e.g. variant), your exploration might not be very smooth and you may underperform.

I would do few (~1000 per arm) Thomson sampling iterations at each batch and allocate the traffic given the proportion of win in this few iterations. I don't have a theory to support this idea, but it seems like it keeps intact the core idea of Thomson sampling (i.e. pull the arm with a frequency equals to its probability to be the best given the data available).

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.