Okay, so, in the traditional Bernoulli Urn problem, we have an urn with a number N, possibly infinite, of coloured balls, and there are k possible colours. That one I grok.

However, what if I don't actually know what k is? That is, what if I have an urn with N balls and an unknown but finite and strictly positive number of possible colours?

The main question is, in fact, what my priors should be. What's the prior that there is exactly one colour? Exactly two? At least two? How do I update on the relative frequencies of each colour? Is this problem even solvable?

My first lines of thinking are to have a vector of parameters $\vec \theta \in \mathbb R^\infty$ such that the first parameter is the number of colours in the urn (let's call it $\alpha$) and the remaining are the relative frequencies of each colour. If $P(A=n|\vec\theta)$ is the probability that the colour of the next draw will be n given the knowledge contained by $\vec\theta$, we'd have:

  • $\vec\theta = (\alpha, p_1, p_2, p_3, ...)$
  • $\alpha \in \mathbb N^*$
  • $\left(\sum\limits_{n=1}^\infty P(\alpha = n) \right)= 1$
  • $\left(\sum\limits_{n=1}^\infty p_n\right) = 1$
  • $\forall n > \alpha : p_n = 0$
  • $\forall n \in \mathbb N^* : P(A=n|\vec\theta) = p_n$

However, this is just wild speculation on my part. I'm mostly curious about whether this is even in principle solvable. What I'd want to know is a way to compute both the prior (objective/uninformative) and posterior distributions of $P(\vec\theta)$ or, in other words, the pdfs $P(\alpha)$, $P(p_1)$, $P(p_2)$, etc. How to start with them and how to update on them.

  • $\begingroup$ "what my priors should be" -- we can't tell you your own priors. You tell us what your prior distribution over the k's are; this will depend in the exact details of the situation you're in (not just the part you describe). $\endgroup$
    – Glen_b
    Commented Dec 6, 2013 at 5:57
  • $\begingroup$ And if the part I describe is exactly the totality of the situation I'm in? Also, how about the posteriors, how do I compute them? $\endgroup$
    – Red
    Commented Dec 6, 2013 at 7:59
  • $\begingroup$ With all information, the priors would still be subjective - the additional information would inform your priors. $\endgroup$
    – Glen_b
    Commented Dec 6, 2013 at 8:55
  • $\begingroup$ So there is no way to calculate an objective/uninformative prior, then? And that still doesn't answer the question about how to calculate the posterior. $\endgroup$
    – Red
    Commented Dec 6, 2013 at 9:27
  • 2
    $\begingroup$ But how can I tell you to use an uninformative prior, unless you said you wanted one? $\endgroup$
    – Glen_b
    Commented Dec 6, 2013 at 10:18


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.