Poisson likelihood and zero counts in expected value

Question

The following lines are taken from a lecture found here. I have a very small question regarding this explanation of Poisson likelihood.

The Poisson likelihood can be used any time your data come in discrete intervals (which we'll call “counts”), and the counts are independent of each other. Schematically, we imagine dividing data space up into “bins”, which could be bins in energy channel of our detector, location on the sky, time of arrival, or any of a number of other things. Suppose that in a particular model $m$, you expect there to be $m_i$ counts in bin $i$. Then if the model is correct the likelihood of actually observing $d_i$ counts in bin $i$ of the data is, from the Poisson distribution,

$L_i = m_i^{d_i}/d_i! \; exp(-m_i)$

Note that $m_i$ can be any positive real number, whereas $d_i$ must be an integer. Note also that the sum of $L_i$ from $d_i = 0$ to ∞ is 1. The likelihood for the whole data set is the product of the likelihoods for each bin:

$L = \Pi\; m_i^{d_i}/d_i! \; exp(-m_i)$

This becomes better and better approximated by a Gaussian as $m_i$ increases. (...) If one uses Poisson likelihoods, small numbers are fine. In fact, if you can manage, the best way to represent your data is to have bins so tiny that you expect either 0 or 1 count per bin.

The question I have is: what happens if one of the expected values $m_i$ is zero (ie: the model predicts zero counts in that bin $i$)? Wouldn't that make the final $L$ value null?

Edit

After reading @leonbloy and @Henry's answer I think I should explain a bit more my particular problem with this analysis and what it is that I'm clearly not understanding.

In my case I have two histograms representing the modeled data and the observed data. So the $[d_i, \; i=1,n]$ histogram is made out of actual observations (ie: the counts in each bin $d_i$ are obtained through an experiment) and the histogram $[m_i, \; i=1,n]$ is produced by a model: this would mean that the $m_i$ counts in my case are integers that can actually take the value 0.

Now, this is where I'm clearly getting something wrong because I read in that article a line that says "Note that $m_i$ can be any positive real number" and @leonbloy's answer also points in that direction, when in my case the $m_i$ values are all integers and do take the value 0 a lot of times.

I must be skipping a crucial step concerning the calculation of the $m_i$ values. So how should I calculate the $m_i$ values if my model gives me counts per bin?

@Henry's answer seems to contradict @leonbloy's answer since he says $m_i=0$ is a possibility whereas @leonbloy says it is not.

In any case, Henry seems to be assuming the whenever $m_i=0$ then $d_i=0$ also so that one obtains $L_i=1$ because $0⁰=1$. This is not my case, since $m_i$ could be 0 but it doesn't mean $d_i$ is forced to be 0 also (my observed histogram, $[d_i , \; i=1,n]$, and my modeled histogram, $[m_i , \; i=1,n]$, are derived independently of each other)

Edit 2

I think I'm beginning to understand, but I'm still a bit lost. Please take a look at this excerpt taken from this paper:

(...) Therefore the selected test to perform on these histogram is a log likelihood test for Poisson statistics (Eidelman et al. 2004):

$-2ln(\lambda(\theta))=2\, \sum\limits_{i=1}^N \, (\nu_i(\theta)-n_i+n_i\, ln\frac{n_i}{\nu_i(\theta)})$ (8)

In this formula $\theta$ is the set of unknown parameters one wants to derive, $n = (n_1 , n_2, . . . , n_N)$ is the data vector containing the observations with $N$ the number of bins in a histogram. $\nu$ are the expected values, which are derived from the histograms of the modelled data and are therefore dependent on $\theta$. When $n_i = 0$, the last term in Eq. (8) is set to zero (...)

As you can see, the analysis is the same (the difference here is that the last equation shows the likelihood ratio), now the expected values are $\nu_i(\theta)$.

From what I've understood so far that parameter $\nu_i(\theta)$ should be independent of the number of counts in the bins and have just one value for each histogram. If this is so then why use the $i$ subscript? And how should I calculate that value for each histogram?

Answer 1

I think that the sentence is poorly worded, one should not use the word "expect" in that context, it can induce confusion. Because to "expect either 0 or 1 count per bin" does NOT mean that "the expected value per bin is either 0 or 1 ($m_i=0$ or $m_i=1$)". It rather means that the expected value per bin is quite small ($m_i$ is quite below 1 - but not zero!), so that the probability of $d_i$ being 0 or 1 is high (or put in other way, the likelihood of the counting being greater than 1 is small).

For example, if $m_i=0.2$, we'd have a Poisson with probabilites $L_0 = 0.8187$, $L_1=0.1638$, so the probability that we have "either 0 or 1 count per bin" is high (0.9825); and that's what the (confusing) expression "we expect either 0 or 1 count per bin" means.

Answer 2

If $m_i=0$ then the likelihood $L_i$ of observing $0$ is $1$ and of observing anything else is $0$.

This is an example of the wider convention where $0^0$ is taken to be $1$ so $\frac{0^0}{0!}\exp(-0)=1$

Answer 3

The statement "Suppose that in a particular model $m$, you expect there to be $m_i$ counts in bin $i$. Then if the model is correct the likelihood of actually observing $d_i$ counts in bin $i$ of the data is, from the Poisson distribution,*

$L_i = m_i^{d_i}/d_i! \; exp(-m_i)$"

is incorrect in a couple of ways. First, $L$ is not the formula for the Poisson distribution. The Poisson has one parameter, let us label it $\lambda$, not one parameter $m_i$ for each bin $i$. The parameter $\lambda$ is the expected value of a random variable from the distribution, not the expected value of the number of counts in a specific bin. Also, the use of $d_i$ is confusing. Here's the correct probability for a single draw from a Poisson to equal $i$:

$L_i = \frac{\lambda^{i}}{i!} \exp(-\lambda)$

Note that $L_i > 0 \space \forall i \ge 0$. If we make $N$ draws from a Poisson, $NL_i$ can be interpreted as the expected number of draws which will have value $i$. This is the same as $m_i$ in the initial statement. As $L_i > 0$, $m_i > 0$ as well. $m_i$ can be any positive real number, as in the initial statement.

The initial statement would seem to lead more naturally to a multinomial distribution, where the expected number of counts in a bin divided by the total number of counts is indeed a parameter, and there is one such for each bin. When reduced to just "counts in bin $i$ vs. counts not in bin $i$" the multinomial simplifies to a binomial; the probability of seeing $d_i$ counts in bin $i$ given $N$ total counts and an expected number of counts in bin $i$ of $m_i$ is:

$L_i = \binom {N}{d_i}(m_i/N)^{d_i}(1-m_i/N)^{N-d_i}$

I'm a little hesitant to post this, because one doesn't expect lecture notes to be this wrong, which implies a significant chance that my brain has malfunctioned and I've just missed something myself.