Skip to main content
Cross Validated

Return to Question

Notice removed Draw attention by Wrzlprmft
Bounty Ended with J. Delaney's answer chosen by Wrzlprmft
added 162 characters in body
Source Link
Wrzlprmft
Wrzlprmft
  • 2.4k
  • 1
  • 20
  • 38

Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. The loaded coins are not identical, but their heads ratio $θ$ follows an unknown distribution $p(θ|\mathcal{M}_\text{loaded})$ obeying some constraints (see below). For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

  • For each coin: the number of heads from hundred tosses (and thus an estimator $\hat{θ}$ for the heads ratio $θ$).
  • Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
  • The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
    • symmetric around ½,
    • smooth,
    • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<\hat{θ}<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

  • Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

  • If yes, is there a name or reference for this approach?

  • If no, is there a better way to determine parameter priors for $\mathcal{M}_\text{loaded}$? I am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.

Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

  • For each coin: the number of heads from hundred tosses (and thus an estimator $\hat{θ}$ for the heads ratio $θ$).
  • Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
  • The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
    • symmetric around ½,
    • smooth,
    • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<\hat{θ}<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

  • Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

  • If yes, is there a name or reference for this approach?

  • If no, is there a better way to determine parameter priors for $\mathcal{M}_\text{loaded}$? I am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.

Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. The loaded coins are not identical, but their heads ratio $θ$ follows an unknown distribution $p(θ|\mathcal{M}_\text{loaded})$ obeying some constraints (see below). For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

  • For each coin: the number of heads from hundred tosses (and thus an estimator $\hat{θ}$ for the heads ratio $θ$).
  • Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
  • The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
    • symmetric around ½,
    • smooth,
    • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<\hat{θ}<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

  • Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

  • If yes, is there a name or reference for this approach?

  • If no, is there a better way to determine parameter priors for $\mathcal{M}_\text{loaded}$? I am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.

added 16 characters in body
Source Link
Wrzlprmft
Wrzlprmft
  • 2.4k
  • 1
  • 20
  • 38

Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

  • For each coin: the number of heads from hundred tosses (and thus an estimator $\hat{θ}$ for the heads ratio $θ$).
  • Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
  • The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
    • symmetric around ½,
    • smooth,
    • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<θ<0.6$$0.4<\hat{θ}<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

  • Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

  • If yes, is there a name or reference for this approach?

  • If no, is there a better way to determine parameter priors for $\mathcal{M}_\text{loaded}$? I am am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.

Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

  • For each coin: the number of heads from hundred tosses (and thus an estimator for the heads ratio $θ$).
  • Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
  • The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
    • symmetric around ½,
    • smooth,
    • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<θ<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

  • Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

  • If yes, is there a name or reference for this approach?

  • If no, is there a better way to determine parameter priors for $\mathcal{M}_\text{loaded}$? I am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.

Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

  • For each coin: the number of heads from hundred tosses (and thus an estimator $\hat{θ}$ for the heads ratio $θ$).
  • Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
  • The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
    • symmetric around ½,
    • smooth,
    • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<\hat{θ}<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

  • Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

  • If yes, is there a name or reference for this approach?

  • If no, is there a better way to determine parameter priors for $\mathcal{M}_\text{loaded}$? I am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.

added 48 characters in body
Source Link
Wrzlprmft
Wrzlprmft
  • 2.4k
  • 1
  • 20
  • 38

Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

  • For each coin: the number of heads ratio $θ$ from hundred tosses (and thus an estimator for the heads ratio $θ$).
  • Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
  • The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
    • symmetric around ½,
    • smooth,
    • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<θ<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

  • Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

  • If yes, is there a name or reference for this approach?

  • If no, is there a better way to determine parameter priors for $\mathcal{M}_\text{loaded}$? I am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.

Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

  • For each coin: the heads ratio $θ$ from hundred tosses.
  • Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
  • The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
    • symmetric around ½,
    • smooth,
    • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<θ<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

  • Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

  • If yes, is there a name or reference for this approach?

  • If no, is there a better way to determine parameter priors for $\mathcal{M}_\text{loaded}$? I am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.

Overview

I want to perform a Bayesian model selection on many datasets and use these datasets to determine the required parameter priors.

Example Scenario: Coins

Suppose I have a collection of thousand coins produced by a machine that randomly produces fair and loaded coins. For each coin, I want to decide whether it’s fair using Bayesian model selection with two models $\mathcal{M}_\text{loaded}$ and $\mathcal{M}_\text{fair}$.

I know:

  • For each coin: the number of heads from hundred tosses (and thus an estimator for the heads ratio $θ$).
  • Model priors $p_\text{fair}$ and $p_\text{loaded}$ with $0.1≤p_\text{fair}≤0.9$.
  • The probability density $p(θ|\mathcal{M}_\text{loaded})$ of the heads ratio of the loaded coins obeys the following constraints:
    • symmetric around ½,
    • smooth,
    • not very far from a uniform distribution, say, $0.1 < p(θ|\mathcal{M}_\text{loaded}) < 10$ everywhere.

With all this given, the main information I am lacking for this is a prior for $p(θ|\mathcal{M}_\text{loaded})$. I estimate this by finding a suitable distribution and fitting it to my data for all coins, ignoring coins with $0.4<θ<0.6$, since those have a decent chance to be fair. The rest of the Bayesian model selection is straightforward.

Questions

  • Is this procedure sound? I acknowledge that I use the same data twice. However, the data for a given coin has barely any impact on the parameter priors relevant to its model selection. (I could also exclude the data for the given coin when determining the priors for its analysis, doing a thousand fits instead of just one.)

  • If yes, is there a name or reference for this approach?

  • If no, is there a better way to determine parameter priors for $\mathcal{M}_\text{loaded}$? I am particularly interested in ways that can be extended to a more complex model space as well as higher-dimensional and unbounded parameter spaces.

added 1 character in body
Source Link
Wrzlprmft
Wrzlprmft
  • 2.4k
  • 1
  • 20
  • 38
Notice added Draw attention by Wrzlprmft
Bounty Started worth 100 reputation by Wrzlprmft
edited title
Link
Wrzlprmft
Wrzlprmft
  • 2.4k
  • 1
  • 20
  • 38
added 59 characters in body
Source Link
Wrzlprmft
Wrzlprmft
  • 2.4k
  • 1
  • 20
  • 38
Source Link
Wrzlprmft
Wrzlprmft
  • 2.4k
  • 1
  • 20
  • 38