How do Bayesians gamble? Try to earn the most at this game:


*

*there are two discrete random processes x(t) and y(t) with two unknown continuous distributions, both are IID processes

*at iteration (t-1)
you pick either "x" or "y", to be used in the following iteration t

*at iteration t
both x(t) and y(t) are revealed
if you picked "x" at t-1 your reward is x(t) dollars, otherwise it's y(t) dollars
then you pick "x" or "y" again and the game repeats ...
Frequentist strategy


*

*at iteration t, estimate the mean E[x] and E[y] , averaging the t past samples of x(t) and y(t) respectively

*pick "x" or "y", depending on which of the two random processes has the highest estimated mean
Bayesian strategy?
Help! What would a Bayesian do?
 A: Given that $x(t)$ and $y(t)$ are IID draws from a distribution, and assuming we want to maximize the expected reward, your problem simplifies to essentially:

Given $n$ samples $\{ x(t) \}_{t=1}^n \stackrel{iid}{\sim} P$ and $\{ y(t) \}_{t=1}^n \stackrel{iid}{\sim} Q$ over $\mathbb R$, how do I decide whether $P$ or $Q$ has a higher mean?

Your "frequentist strategy" just estimates the two means with the standard Monte Carlo estimator and picks the higher estimate.
The most obvious Bayesian strategy would assume some prior over the distributions $P$ and $Q$, find a posterior distribution over the means $\mu_P$ and $\mu_Q$, and then choose $x$ if $\mathbb E[\mu_P] > \mathbb E[\mu_Q]$ under those posteriors. If you have some actual priors about what $P$ and $Q$ might look like, then of course you can and "should" use those.
These priors on distributions might be something like

$P = \mathcal N(\mu, \sigma^2)$, where $\mu \sim \mathcal N(0, 100^2)$ and $\log \sigma^2 \sim \mathcal N(0, 100^2)$.

Then the likelihood of any given $x \sim P$ is just the density of that $x$ under the corresponding $P$, so you have a very definite algorithm to follow, although it might take MCMC to compute your decisions depending on the structure of the priors.
This particular form of parametric prior is probably a bad idea (too brittle). A better approach might be nonparametric Bayesian methods, like for example a Dirichlet process. Honestly, I don't know a ton about these approaches; it may be that there's a simple and flexible prior that gives you an estimator nearly as easy to compute as the frequentist one, or maybe not.
In any case, my guess (not knowing anything in particular about this kind of problem) would be that:


*

*A bad prior can cause you do arbitrarily poorly on this game.

*A good prior will give you an advantage over the frequentist approach, but in most "reasonable" situations probably not a huge one.

*In "unreasonable" situations, the Bayesian approach can win arbitrarily. For example, you might know a priori that one of $P$ and $Q$ is $\mathcal N(100, 1)$ while the other is $0.999 \delta_0 + 0.001 \delta_{10^{13}}$, so the one that almost always gives you $0$ has a much higher expected payoff; the frequentist will immediately prefer the normal distribution until the first huge payoff (which they miss), where the very-knowledgable Bayesian will immediately go for the one with the potentially huge payoff.

