# How to set a Bayesian prior on a set with a large but unknown number of elements?

Let us suppose that we are trying to analyze a given starfish. We would like to know which species does the starfish belong to. We have a list of 1000 starfish species, but we know that there is an unneglectable probability that the list is incomplete.

We know that $$p\%$$ of all starfish belong to species $$A_1$$. Next, we email the pictures of the starfish to an expert on marine biology and he says that the starfish can be $$A_1$$, but it is also possible that it is $$A_2$$ or $$A_3$$.

Let us denote by $$A_1, A_2, A_3$$ the statement that the starfish is $$A_1, A_2, \text{or } A_3$$. Let us denote $$A^{+}_2, A^{+}_3$$ the statement that the marine biologist adds $$A^{+}_2, A^{+}_3$$ to the list of candidate species. Finally $$N_n$$ means that the starfish does not belong to any of the species $$A_1, \dots, A_n$$. We would then update our probabilities as follows:

$$P(A_1|A^{+}_{2}A^{+}_{3}) = P(A_1)P(A^{+}_{2}A^{+}_{3}|A_1)\frac{1}{P(A^{+}_{2}A^{+}_{3})} \\ P(A_2|A^{+}_{2}A^{+}_{3}) = P(A_2)P(A^{+}_{2}A^{+}_{3}|A_2)\frac{1}{P(A^{+}_{2}A^{+}_{3})} \\ P(A_3|A^{+}_{2}A^{+}_{3}) = P(A_3)P(A^{+}_{2}A^{+}_{3}|A_3)\frac{1}{P(A^{+}_{2}A^{+}_{3})} \\ P(N_3|A^{+}_{2}A^{+}_{3}) = P(N_3)P(A^{+}_{2}A^{+}_{3}|N_3)\frac{1}{P(A^{+}_{2}A^{+}_{3})} \\$$ Now, here is the thing. We have no reason to believe that the microbiologist is any better at identifying $$A_2$$ rather than $$A_3$$. So we would expect the posterior for both $$A_2$$ and $$A_3$$ to be equal: $$P(A_2|A^{+}_{2}A^{+}_{3}) = P(A_3|A^{+}_{2}A^{+}_{3})$$ From the abovementioned equations, using the chain rule we see that is the case if and only if $$\frac{P(A^{+}_{3}|A^{+}_{2}A_3)}{P(A^{+}_{3}|A^{+}_{2}A_2)} = \frac{P(A^{+}_{2}|A_2)}{P(A^{+}_{2}|A_3)}$$ Now, this still leaves open the question of priors on $$A_2$$ and $$A_3$$. Intuition tells us, that the prior probability should be a small fraction of the probability that the starfish does not belong to any of the species in consideration so far: $$P(A_2|X) = fP(N_1|X) = f(1-p)$$ However, this leads to a strange situation when we try to update the probabilities sequentially, by first updating on $$A^{+}_2$$: $$P(A_2|A^{+}_2) = P(A_2)\frac{P(A^{+}_2|A_2)}{P(A^{+}_2)} = f(1-p)\frac{P(A^{+}_2|A_2)}{P(A^{+}_2)} \\ P(N_2|A^{+}_2) = P(N_2)\frac{P(A^{+}_2|N_2)}{P(A^{+}_2)} = (1-f)(1-p)\frac{P(A^{+}_2|N_2)}{P(A^{+}_2)}$$ and then $$A^{+}_3$$: $$P(A_2|A^{+}_2A^{+}_3) = P(A_2|A^{+}_2)\frac{P(A^{+}_3|A_2A^{+}_2)}{P(A^{+}_3|A^{+}_2)} = f(1-p)\frac{P(A^{+}_2|A_2)}{P(A^{+}_2)}\frac{P(A^{+}_3|A_2A^{+}_2)}{P(A^{+}_3|A^{+}_2)} \\ P(A_3|A^{+}_2A^{+}_3) = P(A_3|A^{+}_2)\frac{P(A^{+}_3|A_3A^{+}_2)}{P(A^{+}_3|A^{+}_2)} = f(1-f)(1-p)\frac{P(A^{+}_2|N_2)}{P(A^{+}_2)}\frac{P(A^{+}_3|A_3A^{+}_2)}{P(A^{+}_3|A^{+}_2)}$$ where we used the assumption that $$P(A_3|A^{+}_2) = f P(N_2|A^{+}_2)$$. Putting these equations together produces: $$f(1-p)\frac{P(A^{+}_2|A_2)}{P(A^{+}_2)}\frac{P(A^{+}_3|A_2A^{+}_2)}{P(A^{+}_3|A^{+}_2)} = f(1-f)(1-p)\frac{P(A^{+}_2|N_2)}{P(A^{+}_2)}\frac{P(A^{+}_3|A_3A^{+}_2)}{P(A^{+}_3|A^{+}_2)}$$ which simplifies to: $$P(A^{+}_2|A_2)P(A^{+}_3|A_2A^{+}_2) = (1-f)P(A^{+}_2|N_2)P(A^{+}_3|A_3A^{+}_2)$$ and using our necessary and sufficient condition derived from the equality of the posteriors: $$\frac{P(A^{+}_2|A_3)}{P(A^{+}_2|A_3)}\frac{P(A^{+}_2|A_2)}{P(A^{+}_2|N_2)} = (1-f)\frac{P(A^{+}_3|A_3A^{+}_2)}{P(A^{+}_3|A_2A^{+}_2)} \\ P(A^{+}_2|N_2) = \frac{P(A^{+}_2|A_3)}{1-f}$$ Therefore unless we set $$P(A^{+}_2|N_2)$$ as equal to a multiple of $$P(A^{+}_2|A_3)$$ we will get the unintuitive result that the posteriors of $$A_2$$ and $$A_3$$ are not equal. This would be very odd. It would be as if it mattered in which order $$A_2$$ and $$A_3$$ appear in the biologist's email.

This model is quite inflexible, since we cannot set $$P(A^{+}_2|N_2)$$ at liberty, but must always fix it as a multiple of $$P(A^{+}_2|A_3)$$, which limits our modeling capacities. Furthermore we have to be careful with setting $$f$$, since the probabilities have to be smaller than 1.

Given that this model suffers from these restrictions, what would be a better way of setting a prior on a finite, very large, but unknown set?

• There is a whole Bayesian literature on estimating the number of species, see, e.g., Lijoi et al. – Xi'an Apr 15 at 15:54