2
$\begingroup$

What is the theoretical reason why Thompson Sampling needs to involve posterior distributions? Why can we not sample over predictive distributions? (or is the issue that predictive frequentist distributions are difficult to obtain?)

$\endgroup$
  • $\begingroup$ What's really the difference between the distribution collected as updated priors / posteriors vs "frequentist" distribution? The way I see it, it's the difference between an online mean ($\mu\leftarrow\mu+(x_t-\mu)/t$) vs a batch mean ($\mu=\frac1n\sum_{t=1}^nx_t$), they give the same answer. $\endgroup$ – Kris Feb 1 '19 at 3:50
  • $\begingroup$ From what I can tell, the idea is that posterior distribution gives you probability matching, but it has been difficult for me to find literature on why or why not the predictive distribution gives you the same. What's the difference in posterior distributions vs predictive distributions? Outside of the base trivial case as you have mentioned? They can differ quite a bit. $\endgroup$ – Jenny Yang Feb 2 '19 at 5:28
  • $\begingroup$ Great question. Nice answer below by JP, and the whole thing as been viewed 158 times in 8 months. I just don't get this site. $\endgroup$ – Ben Ogorek Sep 15 '19 at 1:17
2
$\begingroup$

A "frequentist" alternative to Thompson Sampling is Bootstrap Thompson Sampling (BTS). The general idea is to approximate sampling from a posterior distribution by using a bootstrap distribution. In this paper by Eckles and Kaptein they use an online "double or nothing" bootstrap. You get the nice benefit of constant time updates regardless of if your posterior distribution is computable or not, so no MCMC necessary for distributions without a closed form.

You also may be interested in this paper by Lihong Li. He derives "generalized Thompson sampling" and makes the suggestion that the performance of Thompson sampling is not due to its Bayesian nature, but rather that the Bayesian perspective on Thompson sampling is just a special case of a more general exponential update algorithm.

| cite | improve this answer | |
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.