1
$\begingroup$

Preliminaries

My question came about when reading Lawrence and Chromy's paper on Maximum Likelihood (ML) estimation of histograms (link). I am aware of this question, but I fail to see how that helps me. Please provide additional info if you were to link me to this answer. In their supplementary information L&C state

$n$ is large enough that the probability that an event is with in any bin is small, leading to a Poisson distribution rather than a multinomial distribution.

By $n$ they mean the number of histogram bins. Why would they even mention that? Here is my trouble with that statement:

Histogram as Multinomial

They observe independent identically distributed random events and bin them in a histogram with $n$ bins with index $j=1,...,n$. Let

$prob(b_j|\theta)$: Probability of observing an event in bin with index $j$.

This is given by a theoretical model that depends on parameter(s) $\theta$ to be estimated by ML. Furthermore let

$$\sum_{j=1}^n prob(b_j|\theta) = 1$$

independent of $\theta$, which is relevant to my question. Then (I think) the likelihood of observing a histogram with $m_j$ counts in bin $j=1,...,n$ is given by a multinomial distribution

$$L_M(\theta) := prob(\{m_j\}|\theta) = \frac{M!}{\prod_{j=1}^n m_j!}\prod_{j=1}^n prob(b_j|\theta)^{m_j}$$

which gives an expected value $\mu_j(\theta)=M\cdot prob(b_j|\theta)$ for each bin, where $M=\sum_{j=1}^n m_j$ is the total number of counts in the histogram.

Poisson Distribution

Lawrence and Cromy model the number of counts in each bin as a Poisson Distribution. The likelihood of observing $m_j$ counts in one specific bin $j$ is

$$prob(m_j|\theta) = \frac{\mu_j(\theta)^{m_j}}{m_j!}\exp(-\mu_j)$$

Now by the independence they can write the likelihood of the set $\{n_j\}$ as

$$L_P(\theta) := prob(\{m_j\}|\theta) = \prod_{j=1}^n\frac{\mu_j(\theta)^{m_j}}{m_j!}\exp(-\mu_j)$$.

My Trouble

This is where I get confused. Why don't L&C just do this calculation:

$$\begin{eqnarray}L_P(\theta)&=& \prod_{j=1}^n\frac{\mu_j(\theta)^{m_j}}{m_j!}\prod_{j=1}^n\exp(-\mu_j(\theta)) \\ &=&\prod_{j=1}^n\frac{\mu_j(\theta)^{m_j}}{m_j!}\exp\left(-\sum_{j=1}^n\mu_j(\theta)\right) \\ &=& \exp(-M)\cdot \prod_{j=1}^n\frac{\mu_j(\theta)^{m_j}}{m_j!} \end{eqnarray}$$ and ultimately after disregarding terms that are not dependent on $\theta$

$$L_P(\theta)\propto \prod_{j=1}^n prob(b_j|\theta)^{m_j}$$

which is analogous to the multinomial likelihood where also $$L_M(\theta)\propto \prod_{j=1}^n prob(b_j|\theta)^{m_j}$$

So maximizing either likelihood should give the same ML estimate for $\theta$...

My questions are

  • Is my reasoning flawed?
  • Where is the difference between using the Multinomial and the Poisson model here?
  • Why would L&C require the probability of an event occuring in any one bin being small?

All help is greatly appreciated.

$\endgroup$
1
$\begingroup$

1. $$L_M(\theta) := prob(\{m_j\}|\theta) = \frac{M!}{\prod_{j=1}^n m_j!}\prod_{j=1}^{n-1} prob(b_j|\theta)^{m_j}\left(1-\sum_{i=1}^{n-1}prob(b_j|\theta)\right)^{m_n}$$

  1. Two distributions are different. One specific difference is we assume that $m_j$ are independent if $m_j$ follows Poisson, and it is reasonable in most real world. But if $m_i$ follows multinomial, $m_j$ and $m_i$ negative correlation.

  2. By increasing n, the probability of an event occuring in any one bin being small, such that multinnomial distribution is very similar to Poisson distribution. For examaple, the negative correlation between $m_i$ follows multinomial, $m_j$ and $m_i$ decreases to ignorable degree.

$\endgroup$
4
  • $\begingroup$ Thanks, I think I almost get it. I feel that the correlation of the $\{m_j\}$ is the key fact I missed. Could you clarify some points: I have never seen the functional form of the multinomial distribution like that. Where does that come from? Is that to incorporate the correlation of the counts? $\endgroup$
    – geo
    Oct 24 '18 at 6:36
  • $\begingroup$ Especially: what does $m_n$ refer to in the sum and where does the $(1-\sum ...)$ term come from? $\endgroup$
    – geo
    Oct 24 '18 at 7:18
  • $\begingroup$ In my equation, there are m-1 parameters, and in yours, there are m parameters. Of course, $m_n$ can be $\sum_{i=1}^n - \sum_{i=1}^{n-1}$. For multinomial, your formula is OK, but need to add two conditions, that sum of probabilities = 1 and sum of $m_i$ = m$(total # of trials). Otherwise, you will go wrong in math proof. $\endgroup$
    – user158565
    Oct 24 '18 at 12:49
  • $\begingroup$ Thanks, I get it now. I got confused with my own notation. $\endgroup$
    – geo
    Oct 24 '18 at 18:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.