Bayesian inference for a model that is a Poisson process

Question

Suppose you have some data, represented by the vector $\bf{y}$, and you have a model that maps parameters in vector $\bf{x}$ to $\bf{y}$:

$$f(\bf{x}) = \bf{y}$$

Suppose that $f$ is actually a Poisson process. So given $\bf{x}$, you can imagine sampling a poisson distribution to compute $\bf{y}$. This is a stochastic process, such that evaluating $f(\bf{x})$ two times with the same $\bf{x}$ can yield a different $\bf{y}$, because you are sampling a Poisson distribution.

Now suppose given data $\bf{y}$, you want to do a Bayesian inversion for the $\bf{x}$ parameters. To do this with a method like MCMC you must evaluate the the likelihood function, $p(\bf{y} | \bf{x})$, which involves computing $f(\bf{x})$, so therefore the likelihood function is a Poisson process.

How do you do this inference problem? Is it OK to just code this up using a MCMC algorithm like emcee such that $p(\bf{y} | \bf{x})$ samples poisson distributions, and therefore has some randomness? Or is there a better way to do the problem?

Edit: I’m most uncomfortable with a likelihood function that is not deterministic / contains randomness. Is this ok?

Edit 2: I’ve realized this question is not very well explained, so I have posted a new question: Bayesian inversion with a stochastic likelihood function

Answer 1

We use the relationship:

$$f(\theta;x) \propto f_X(x;\theta)f_{\Theta}(\theta)$$

which is based on the definition of conditional probability. I am using the more standard notation in which $x$ is the data and $\theta$ is the parameter.

In your case, $f_{\Theta}(\theta)$, your prior distribution, is Uniform. If it's Uniform over an interval of finite length, then it is a proper prior, as it integrates to one, otherwise it's an improper prior, because it won't. In some cases this will cause your posterior to be improper as well. I'll assume that it's Uniform over the positive real line in what follows. The fact that it's Uniform means it's a constant, and we can ignore it - except for the potential problems that impropriety can cause. The reason we can ignore it is that at the end of the process we are going to have to determine the constant of integration anyway - the constant that ensures our distribution will integrate to one - so including constant terms in our work at this stage is pointless.

Let's work through the math.

$$ \begin{eqnarray} f_X(x;\theta) &=& e^{-n\theta}{\theta^{\sum x_i} \over x!} \\ f(\theta;x) &\propto& e^{-n\theta}\theta^{\sum x_i} \end{eqnarray}$$

Does this look like a well-known distributional form? Yes, the Gamma distribution, with shape parameter $\sum x_i + 1$ and rate parameter $n$. Since this distribution isn't improper, we know our improper prior hasn't caused us any problems.

If, on the other hand, your prior is Uniform over an interval, say $[a,b]$, then we need to make a slight alteration. Instead of:

$$f(\theta;x) = e^{-n\theta}\theta^{\sum x_i}{n^{\sum x_i + 1} \over \Gamma(\sum x_i + 1)}$$

we will have:

$$f(\theta;x) = {e^{-n\theta}\theta^{\sum x_i} \over \int_a^be^{-n\theta}\theta^{\sum x_i}d\theta}$$

Note that the two forms are actually the same; the first one is what you get when $a = 0$ and $b = \infty$.