1
$\begingroup$

My question is complementary to a previous problem : Bayesian inference on binarized Poisson distribution. I retake the previous notations. Problem description :

I am counting the number of balls falling in T boxes.

X=[0, 2, 1, 0, 0, 2, 1, 0, 2, 1, 1, 0, 0, 2, 0, 0, 1, 1, 0, 0]

$\forall t \in [1, \dots, T], X_t$ is drawned as a superposition of N independant Poisson processes with intensities $\lambda_i$. Thus, $X_t \sim Poisson(\sum_{i=1}^{N} \lambda_i).$ So : $P(X_t = k) = \frac{(\sum_{i=1}^{N}\lambda_i)^{k}e^{-\sum_{i=1}^{N}\lambda_i}}{k!}$ since a sum of poissons is poisson.

Then,

I binarized the sequence to obtain $X_1$ like in the link example by :

$\forall t \in [1, \dots, T], X_1(t) = 1 \quad if \quad X(t) \geq 1$, 0 otherwise.

Then, X_1=[0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 1, 0, 0, 1, 0, 0, 1, 1, 0, 0]

If I am right $X_1(t) \sim Bernoulli(p = P(X_t \geq 1)) = Bern(p = 1 - P(X_t = 0)) = Bern(p=1-e^{-\sum_{i=1}^{N}\lambda_i})$

Finally, I would have a likelihood function with a Bernoulli form but a kind of Poisson parameter:

$P(X_1 | \lambda) = \displaystyle \prod_{t=1}^{T} (1-e^{-\sum_{i=1}^{N}\lambda_i})^{X_1(t)} (e^{-\sum_{i=1}^{N}\lambda_i})^{1-x_1(t)}$

My question : Is it possible to learn the $\lambda_i$ intensities of superposed Poisson Processes through Bayesian Inference and more specifically Gibbs sampling? Which kind of prior on $\lambda_i$ would you use to account for the exponential parameter of the bernoulli distribution and still be able to recognize a samplable posterior ? (Gamma prior seemed appealing to me even if beta distribution is conjugate with bernoulli...) Would you have any reference for estimating intensities of superposition of poisson processes ?

Thx for you help & time my dear statistician friends.

$\endgroup$
6
  • 1
    $\begingroup$ Have you noticed that all your probabilities depend only on the sum of the $\lambda_i$? That is telling you that the data provide information only about the sum and not about the individual $\lambda_i.$ $\endgroup$
    – whuber
    Commented Jun 29, 2022 at 15:22
  • $\begingroup$ "Is it possible to learn the λi intensities of superposed Poisson Processes through Bayesian Inference and more specifically Gibbs sampling?" Yes it is possible, but why would you like to do this? This seems like an XY problem. $\endgroup$ Commented Jun 29, 2022 at 21:00
  • $\begingroup$ "Which kind of prior on λi would you use to account for the exponential parameter of the bernoulli distribution and still be able to recognize a samplable posterior ? " with a beta prior you even have posterior that can be described analytically. $\endgroup$ Commented Jun 29, 2022 at 21:06
  • $\begingroup$ "Would you have any reference for estimating intensities of superposition of poisson processes" en.m.wikipedia.org/wiki/… $\endgroup$ Commented Jun 29, 2022 at 21:07
  • $\begingroup$ Correction: the conjugate posterior for this case is not a beta distribution (which would be for the one-dimensional case) but instead a Generalized Dirichlet distribution. $\endgroup$ Commented Jun 30, 2022 at 5:44

1 Answer 1

2
$\begingroup$
  1. I'd pick the prior that best represents my understanding of the underlying mechanisms of the problem. There is a lot you haven't told us, so it is next to impossible for me to guess a meaningful prior. Who knows? Perhaps $\lambda_i \ne 3$ for some righteous reason. Fruitless for me to speculate.

  2. Failing that, the problem devolves to one of selecting a prior that respects (if not actually celebrates) ignorance, while still holding on to a few crumbs from your data.

  3. Following @whuber's hint, let's find $\Lambda= \sum_{i=1}^N \lambda_i$.

  4. And let's clarify and make explicit: the index $t$ describes independent random samples conditional on $\Lambda$. So knowledge of the measurement $X_t=k_t$ gives no information about $X_s=k_s, (s \ne t)$ given $\Lambda$, viz., $P(X_t=k_t,X_s=k_s|\Lambda)=P(X_t=k_t|\Lambda)P(X_s=k_s|\Lambda).$

  5. Assuming independent measurements in $t$, then we can now define $$ \bar{\lambda}_{NT}=\mathbb{E}_t[\mathbb{E}_i [\lambda_i]] = \frac{1}{NT}\sum_{t=1}^T X_{t}, ~~~~\text{where } X_t = \sum_{i=1}^N X_{t,i}. $$

  6. We can use the following assertions:

  • $\lambda_i \ge 0$ for all $i$.
  • $\bar{\lambda}_{NT}\ge 0$.
  • There is no natural upper limit of $\lambda_i$.
  • There is no evidence that $\lambda_i$ is distributed differently from $\lambda_j$.
  1. If there's nothing else, then we can invoke maximum entropy on a prior distribution for $\lambda_i$ with a known lower bound, no upper bound, and a (nearly constant) mean $\bar{\lambda}_{NT}$ and assert to that $\lambda_i \sim \text{Exp}(1/\Lambda)$, where the hyperparameter $\Lambda$ can have a simple gamma empirical Bayes hyperprior $\pi(\Lambda) = \text{gamma} ( \alpha=\bar{\lambda}_{NT}+1,\beta=1)$ so that your data is consistent with the mode of the of the hyperprior $\Lambda=\bar{\lambda}_{NT}$. Lots of other noninformative hyperpriors are also possible.

  2. Caveat: maximum entropy uses a hard constraint that $\mathbb{E}[\lambda_i]=$const. Here $\bar{\lambda}_{NT}$ is a sample mean of $N$ samples, and his form is asymptotically valid as $NT \rightarrow \infty$. But if $T$ is large and finite, the prior is a narrow mixture of exponentials. If the range of $\bar{\lambda}_{NT}$ is narrow enough, you are looking at a very narrow mixture of exponential distributions, which is approximately a simple exponential.

Are you comfortable with the idea that maximum entropy captures the problem? If you aren't comfortable doing this, then a-ha! there is something you aren't telling us!


Addendum: I haven't addressed the question of "binarized" data because I don't understand the motivation for that (de)generative process. Just before you binarize, you have the full data set: $X_1, \dots, X_T$. Since $\bar{\lambda}_{NT}$ comes from this, and since your interest is in finding a distribution for $\lambda_i$. If you really only want to work with 0s and 1s, well, perhaps we can estimate $\bar{\lambda}_{NT}$ from that, but it would be inferior. Either way, the maximum entropy method seems to be the only way forward, unless there's more you can say.

$\endgroup$
2
  • $\begingroup$ 1. the $\lambda_i$'s belongs to $|R^{+*}$. Thus, in my mind a gamma distribution seemed appropriated for its definition domain. 4. 5. I totally agree 6. yes 8. I need to check since I am not very familiar with maximum entropy principle. Addendum : Actually, this problem is a small part of the problem I am currently working on and I didn't mention some information. In my entire problem, $X$ is a matrix of boxes with balls. At each column, the poisson random variables identity that contribute to the number of balls are known actually. Their intensities need to be estimatee. Thx a lot. $\endgroup$
    – user361947
    Commented Jun 30, 2022 at 12:14
  • $\begingroup$ A gamma distribution would be an OK prior for the reasons you mention. Exp$(\beta)=$gamma$(\alpha=1,\beta)$, after all. But you have to chose a prior based on what you know, not what looks good. I'm guessing that $\bar{\lambda_i}$ is some knowable constant. If var$(\lambda_i)$ is also knowable constant, then maxent gives a different solution, which would be Normal if $-\infty < \lambda_i < \infty$, but no it isn't normal. You may wish to peruse en.wikipedia.org/wiki/Maximum_entropy_probability_distribution for other common constraints and the distributions they yield. $\endgroup$ Commented Jul 1, 2022 at 1:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.