Okay, so, in the traditional Bernoulli Urn problem, we have an urn with a number N, possibly infinite, of coloured balls, and there are k possible colours. That one I grok.
However, what if I don't actually know what k is? That is, what if I have an urn with N balls and an unknown but finite and strictly positive number of possible colours?
The main question is, in fact, what my priors should be. What's the prior that there is exactly one colour? Exactly two? At least two? How do I update on the relative frequencies of each colour? Is this problem even solvable?
My first lines of thinking are to have a vector of parameters $\vec \theta \in \mathbb R^\infty$ such that the first parameter is the number of colours in the urn (let's call it $\alpha$) and the remaining are the relative frequencies of each colour. If $P(A=n|\vec\theta)$ is the probability that the colour of the next draw will be n given the knowledge contained by $\vec\theta$, we'd have:
- $\vec\theta = (\alpha, p_1, p_2, p_3, ...)$
- $\alpha \in \mathbb N^*$
- $\left(\sum\limits_{n=1}^\infty P(\alpha = n) \right)= 1$
- $\left(\sum\limits_{n=1}^\infty p_n\right) = 1$
- $\forall n > \alpha : p_n = 0$
- $\forall n \in \mathbb N^* : P(A=n|\vec\theta) = p_n$
However, this is just wild speculation on my part. I'm mostly curious about whether this is even in principle solvable. What I'd want to know is a way to compute both the prior (objective/uninformative) and posterior distributions of $P(\vec\theta)$ or, in other words, the pdfs $P(\alpha)$, $P(p_1)$, $P(p_2)$, etc. How to start with them and how to update on them.
