2
$\begingroup$

I'm implementing Thompson sampling for a multi armed bandit problem (see http://en.wikipedia.org/wiki/Thompson_sampling). The underlying Bayesian model is a Bayesian Linear Regression, which has a conjugate Normal-inverse gamma (NIG) prior for the regression coefficients and the standard error. This NIG distribution is multivariate over the regression coefficients, but I'm not sure how this fits into the Thompson sampling paradigm.

Traditionally, you are supposed to sample from the posterior distribution and choose the sample that maximizes the expected reward (i.e. if you are running a beta-binomial model describing the probability of success, then you chose the highest sample from the beta distributions). But, with a multivariate sample, I'm not really sure how to choose the sample that maximizes the expected reward.

Wondering if anyone who's familiar with Thompson sampling or multi-armed bandits could help. Thanks!

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Typically a bandit problem is framed as minimizing regret, which is the sum of the difference between your cumulative rewards and the rewards of a policy that knows the underlying distribution from the outset. This implies that you already have a reward function which maps the multivariate distribution to a scalar reward, which is what you use to pick the next action in the Thompson sampling approach. $\endgroup$
    – combo
    Jan 16, 2017 at 18:44

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.