Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

• For each coin: the heads ratio $θ$ from hundred tosses.
• Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
• The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
  • symmetric around ½,
  • smooth,
  • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<θ<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

• Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

• If yes, is there a name or reference for this approach?

• If no, is there a better way to determine parameter prior for $\mathcal{M}_\text{loaded}$? I am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.