This might be a very broad question but I'm wondering whether we can say a model of Bayesian inference is "correct" or not about assuming independence.

For example, suppose there are $N$ coins each of which is going to be tossed for $M$ times.

The probability of Head of coin $i$ is denoted by $p_i$.

There are two statisticians: Alice and Bob.

Alice somehow thinks coins are not correlated. So, she sets up a model that takes the coin-tossing of coin $i$ independent from coin-tossing of coin $j$. For example, she assumes $$p_i\sim Beta(a_i,b_i)$$ and then updates her posterior independently across $i$.

On the other hand, Bob believes coins might be correlated each other, so he sets up a more complicated model that can accommodate possible correlations. For example, he assumes possible outcomes are $(H,H,\cdots,H)$ to $(T,T,\cdots,T)$ with corresponding probabilities $p=p_{(H,\cdots,H)},\cdots, p_{(T,\cdots,T)}.$

Assuming $p\sim Dir(a)$ for some vector of parameter $a$, he updates his posterior based on the outcomes he observes.

My question here, can we say Alice (or Bob) is wrong in building a model in that way? Can we say Alice is wrong because she ignores possible dependency? or can we say Bob is wrong because he's focusing too much on the possible correlations?

I wonder what makes a model wrong or correct when it comes to the independence assumption.

Any help or suggestions will be really appreciated!


I think most would agree every model is wrong, but some are better. I think it may be appropriate to compare their marginal likelihood (or model evidence/normalizing constant) for a more concrete decision.

(See model selection.)


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.