# Bayesian inference on Binomial process under different assumptions

Let me set the context for the idealized question below: The search for extraterrestrial intelligence (SETI) has been listening for radio waves from extraterrestrial sources for half a century, with only negative results. Assume that our own radio broadcasts over the past half century have reached some distance through the galaxy and that if extraterrestrials existed within that volume they would have received our signals and responded back (given sufficient time). But of course we've received no such alien radio signals. In a simplified model, we can state that there are no advanced aliens in the neighborhood defined by our radio "reach," and the import depends upon the number of expected habitable planets in our galaxy (and thus, proportionally, the neighborhood).

Fifty years ago we had estimates of the number of "possibly habitable" planets but every once in a while astronomers discover more habitable planets than we had previously thought. This affects both the number of planets now deemed uninhabitable as well as the number of planets yet to be explored. Those astronomers excite the public into thinking this means the probability of intelligent life in our galaxy has gone up.

But is this true (statistically)? Doesn't the fact that the negative discovery results now apply to a larger number of potential planets mean that the probability of finding intelligent life in the galaxy has gone down? Or does it stay the same? Or does it increase?

And what about "passive" listening? How do our negative results over a larger number of potential inhabitable planets affect our estimate of the probability of finding intelligent life anywhere in the galaxy?

Idealized, simplified question

Consider a long (linear) hotel of known length but assumed number of rooms, $N_r$. Some unknown number of guests, $N_g$, are placed in rooms randomly, and each room can hold an arbitrarily large number of guests; thus we model the placement of guests into rooms as a Poisson process.

There is an unknown probability $p$ that a guest is placed in a room. Initially, we model this unknown probability as coming from a uniform distribution: $0 < p < p_{max} \leq 1$. Initially we assume we know the number of rooms, $N_r$, so this unknown probability $p$ determines the unknown number of guests in the hotel, $N_g$.

Now we stand at one end of the hotel and shout down the hallway thereby reaching a known fraction, $f_1$ of the total rooms. We know that any guest in the closest $f_1 N_r$ rooms will hear our shout and shout back and we will certainly hear them.

Suppose we shout, as just described, but we hear no replies, proving that there are no guests in the rooms that could hear us. Consequently, our estimate of $p$, and thus the number of guests in the hotel, $N_g$, must be revised downward. Clearly as we shout louder, reaching more a larger percentage of the rooms (i.e., increasing $f_1$), and hear no replies, our estimate of $p$ (and thus of $N_g$) gets smaller and smaller. When we shout so loud as to reach the entire hotel ($f_1 = 1$) and hear no replies, we know that $p = 0$, and thus $N_g = 0$.

Suppose we shout, reaching a known fraction $f_1$ of the rooms as just described, hear no replies, and revise our estimate of $p$ (and thus $N_g$). But now someone informs us that actually, our assumed number of rooms was incorrect. In fact, there are more total number of rooms, $N_r$, for the given hotel than we had assumed. Does out estimate of the number of guests in the hotel go up, down, or stay the same? Does the probability $p$ go up, down, or stay the same? What are the rigorous formulas governing such a relation? Are there cases in which the (upward) revised number of rooms in the hotel mean that shouting louder (increasing $f_1$) is less likely to find a guest in the larger number of rooms reached by our louder shout?

A simple illustration

Suppose we have prior assumption that there are just two rooms in the hotel ($N_r =2$)and we have no prior knowledge of the probability a room is filled (or equivalently, the total expected number of guests). We model this knowledge of the as a uniform (Laplace) prior: $P(p = 0)= 1/3, P(p = 1/2) = 1/3$ and $P(p=1) = 1/3$. Now we shout and reach half the total rooms, which in this case is just one room, and hear no response; we are certain that there is no guest in the single closest room. What does this tell us about our estimate of the probabilities and the expected number of guests in the remaining room?

We use Bayes theorem to update our estimate of the probabilities: $P(p|x) \propto P(x|p)P(p)$ where we'll normalize afterwards. Here $x$ denotes the measurement of no guest in the closest half of the rooms, i.e., in the closest room.

• $P(p=0|x) \propto 1 \cdot 1/3$ because $P(x|0) = 1$... i.e., if the probability is $0$, then it is certain we we find no guest in the first room.
• $P(p = 1/2|x) \propto 1/2 \cdot 1/3$ because $P(x|1/2) = 1/2$... i.e., if the probability is $1/2$, then there is a $1/2$ chance we'll find a guest in the closest room, $1/2$ chance we will not find a guest in the closest room.
• $P(p=1|x) \propto 0 \cdot 1/3$ because $P(x|1) = 0$... i.e., if the probability is $1$, then it is impossible to find there is no guest in the closest room.

After normalization, our posterior probabilities are: $P(0) = 2/3, P(1/2) = 1/3$ and $P(1) = 0$. Naturally, after hearing nothing from the closest room we are certain that $p$ does not equal 1.0. But notice that it is twice as likely that $p = 0$ than $p = 1/2$, due to the combinatorics of room fillings.

Under the assumption that there are two rooms, after finding no guest in the first room we expect to find a guest in the remaining room $1/6$ of the time.

Now we are told that in fact there were actually four (not two rooms). With the natural Laplace prior $P(p=0) = 1/5$, P(p = 1/4) = 1/5$, P(p=1/2) = 1/5, P(p=3/4) = 1/5$ and $P(p = 1) = 1/5$. When we repeat this we find that we expect to find at least one guest in the remaining two rooms $13/60$.

Is this change in expected number of guests monotonic in the number of rooms? Are there values of $f_1$ that make the expected number of guests found increase? Decrease? Stay the same?

Is there a specific citation addressing this class of problem (not some general reference on Bayesian inference or statistical estimation)?

• Seems like a very interesting question and I would like to think about it more. I have lost in the second paragraph already. It seems to contradict to the first one that speaks about Poisson whereas the second paragraph seems to be rather about Bernoulli (there is a $p$ that there is a guest in a room, we have $N_r$ rooms, thus $p\cdot N_r$ is the expected number of guests). Please help me to understand this better. Thank you. – Karel Macek Sep 17 '17 at 10:09
• So you have $N_g|N_r,p \sim \text{Binomial}(N_r,p)$, and you put priors on $N_r$ and $p$. You mention that $p \sim \text{Uniform}(0,p_{\text{max}})$, but I'm not getting as clear of a sense what priors you want for $N_r$. Also, how often are you shouting? If you shout once, then you can just reflect that in your prior for $N_r$, and use Bayes' rule straightforwardly. Otherwise, you might want something sequential. Also, are you more interested in predicting $N_g$, or in posterior inference for $N_r$? Last: how are parameters "varying"? You mentioned this word in the title. – Taylor Sep 18 '17 at 16:28
• @Taylor: I've modified my title and writeup. I'm primarily interested in the claim from SETI researchers that finding more habitable planets ("more hotel rooms") always means that the probability of finding intelligent life ("an occupied room", $N_g$) will increase, given assumptions. On the one hand, in the absence of any measurements ("shouts") an increased number of habitable planets would increase the chance that at least one of them is habitable. On the other hand, the negative results of the measurements now applies to a larger number of planets ("rooms"). Which effect dominates? – David G. Stork Sep 18 '17 at 16:57

I'll propose a slightly different model. Let us assume (incorrectly) that all star systems have the same probability $p_h$ of having a habitable planet, and all habitable planets have the same probability $p_l$ of life. In this case, if we sample $M$ star systems, the distribution of the number of habitable planets we observe ($m$) is Binomial($M,p_h$), and the distribution of the observed number of planets with life conditional upon the number of observed habitable planets is Binomial($m, p_l)$. The distribution of the observed number of planets with life given the number of star systems we've observed is therefore Binomial($M, p_hp_l$). (The proof of this is on the Wikipedia page for the Binomial distribution, under "conditional binomials.")
Define $p_L = p_hp_l$. In our case, we've observed one planet with life and $M-1$ planets without, so our MLE of $p_L$ is $1/M$. Clearly, increasing the number of star systems we sample without finding life decreases our estimate of $p_L$. This is true regardless of whether the star systems have habitable planets; the increase in our estimate of $p_h$ is more than offset by the decrease in our estimate of $p_l$.
To see why this is so (at a more detailed level than "it must be so, as $\hat{p}_L$ decreases"), let's consider what happens to our MLEs of $\hat{p}_h$ and $\hat{p}_l$ when we observe one new star system with a habitable planet and no life. Assume that before this observation we have observed $M$ systems with $m$ habitable planets, one of which has life. $\hat{p}_h$ increases from $m/M$ to $(m+1)/(M+1)$, but $\hat{p}_l$ decreases from $1/m$ to $1/(m+1)$. The relative decrease in $\hat{p}_l$ is greater than the relative increase in $\hat{p}_h$, and the product decreases from $1/M$ to $1/(M+1)$, as we expect.
• The the conditions explored in paragraph 3 and my case are different. In paragraph 3 a new star system is observed (and found to be without life). That does NOT change the total number of UNOBSERVED star systems. In my case, changing the prevalence of star systems increases BOTH the number (previously) found to be lifeless AND the number of not-yet-tested systems, which might or might not contain life. Alternatively: Which is "more favorable" for the SETI case: ruling out $1$ out of a galaxy total of $4$ systems, or ruling out $10^6$ out of a galaxy total of $4 \cdot 10^6$ systems? – David G. Stork Sep 18 '17 at 20:07
• In the real world, the number of unobserved star systems is so large that observing one more makes essentially no difference, statistically speaking, in terms of its effect on the number of unobserved systems. Note also that by increasing $p_h$, the estimated number of overall habitable planets increases as well, as it's just, say, $N*p_h$, where $N$ is the total number of star systems.... So the estimated number of habitable but not-yet-tested systems does indeed increase, but the estimated number of actually inhabited systems decreases. I'll extend the answer later on to clarify this. – jbowman Sep 18 '17 at 23:42