Is it possible to estimate the parameters of a superposition of Poisson processes through Bayesian inference from a binarized sequence?

Question

My question is complementary to a previous problem : Bayesian inference on binarized Poisson distribution. I retake the previous notations. Problem description :

I am counting the number of balls falling in T boxes.

X=[0, 2, 1, 0, 0, 2, 1, 0, 2, 1, 1, 0, 0, 2, 0, 0, 1, 1, 0, 0]

$\forall t \in [1, \dots, T], X_t$ is drawned as a superposition of N independant Poisson processes with intensities $\lambda_i$. Thus, $X_t \sim Poisson(\sum_{i=1}^{N} \lambda_i).$ So : $P(X_t = k) = \frac{(\sum_{i=1}^{N}\lambda_i)^{k}e^{-\sum_{i=1}^{N}\lambda_i}}{k!}$ since a sum of poissons is poisson.

Then,

I binarized the sequence to obtain $X_1$ like in the link example by :

$\forall t \in [1, \dots, T], X_1(t) = 1 \quad if \quad X(t) \geq 1$, 0 otherwise.

Then, X_1=[0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0]

If I am right $X_1(t) \sim Bernoulli(p = P(X_t \geq 1)) = Bern(p = 1 - P(X_t = 0)) = Bern(p=1-e^{-\sum_{i=1}^{N}\lambda_i})$

Finally, I would have a likelihood function with a Bernoulli form but a kind of Poisson parameter:

$P(X_1 | \lambda) = \displaystyle \prod_{t=1}^{T} (1-e^{-\sum_{i=1}^{N}\lambda_i})^{X_1(t)} (e^{-\sum_{i=1}^{N}\lambda_i})^{1-x_1(t)}$

My question : Is it possible to learn the $\lambda_i$ intensities of superposed Poisson Processes through Bayesian Inference and more specifically Gibbs sampling? Which kind of prior on $\lambda_i$ would you use to account for the exponential parameter of the bernoulli distribution and still be able to recognize a samplable posterior ? (Gamma prior seemed appealing to me even if beta distribution is conjugate with bernoulli...) Would you have any reference for estimating intensities of superposition of poisson processes ?

Thx for you help & time my dear statistician friends.

Answer 1

1. I'd pick the prior that best represents my understanding of the underlying mechanisms of the problem. There is a lot you haven't told us, so it is next to impossible for me to guess a meaningful prior. Who knows? Perhaps $\lambda_i \ne 3$ for some righteous reason. Fruitless for me to speculate.

2. Failing that, the problem devolves to one of selecting a prior that respects (if not actually celebrates) ignorance, while still holding on to a few crumbs from your data.

3. Following @whuber's hint, let's find $\Lambda= \sum_{i=1}^N \lambda_i$.

4. And let's clarify and make explicit: the index $t$ describes independent random samples conditional on $\Lambda$. So knowledge of the measurement $X_t=k_t$ gives no information about $X_s=k_s, (s \ne t)$ given $\Lambda$, viz., $P(X_t=k_t,X_s=k_s|\Lambda)=P(X_t=k_t|\Lambda)P(X_s=k_s|\Lambda).$

5. Assuming independent measurements in $t$, then we can now define $$ \bar{\lambda}_{NT}=\mathbb{E}_t[\mathbb{E}_i [\lambda_i]] = \frac{1}{NT}\sum_{t=1}^T X_{t}, ~~~~\text{where } X_t = \sum_{i=1}^N X_{t,i}. $$

6. We can use the following assertions:

• $\lambda_i \ge 0$ for all $i$.
• $\bar{\lambda}_{NT}\ge 0$.
• There is no natural upper limit of $\lambda_i$.
• There is no evidence that $\lambda_i$ is distributed differently from $\lambda_j$.

7. If there's nothing else, then we can invoke maximum entropy on a prior distribution for $\lambda_i$ with a known lower bound, no upper bound, and a (nearly constant) mean $\bar{\lambda}_{NT}$ and assert to that $\lambda_i \sim \text{Exp}(1/\Lambda)$, where the hyperparameter $\Lambda$ can have a simple gamma empirical Bayes hyperprior $\pi(\Lambda) = \text{gamma} ( \alpha=\bar{\lambda}_{NT}+1,\beta=1)$ so that your data is consistent with the mode of the of the hyperprior $\Lambda=\bar{\lambda}_{NT}$. Lots of other noninformative hyperpriors are also possible.

8. Caveat: maximum entropy uses a hard constraint that $\mathbb{E}[\lambda_i]=$const. Here $\bar{\lambda}_{NT}$ is a sample mean of $N$ samples, and his form is asymptotically valid as $NT \rightarrow \infty$. But if $T$ is large and finite, the prior is a narrow mixture of exponentials. If the range of $\bar{\lambda}_{NT}$ is narrow enough, you are looking at a very narrow mixture of exponential distributions, which is approximately a simple exponential.

Are you comfortable with the idea that maximum entropy captures the problem? If you aren't comfortable doing this, then a-ha! there is something you aren't telling us!

---

Addendum: I haven't addressed the question of "binarized" data because I don't understand the motivation for that (de)generative process. Just before you binarize, you have the full data set: $X_1, \dots, X_T$. Since $\bar{\lambda}_{NT}$ comes from this, and since your interest is in finding a distribution for $\lambda_i$. If you really only want to work with 0s and 1s, well, perhaps we can estimate $\bar{\lambda}_{NT}$ from that, but it would be inferior. Either way, the maximum entropy method seems to be the only way forward, unless there's more you can say.