Im trying to calculate the likelihood function of my latent state space model.

My model has Poisson observations

$p(y_t|\beta_t;x_t) \sim \mathcal{Poiss}(z)$.

where $z$ is the rate of the poisson process, $z = exp(\beta_t \cdot x_t)$ and $x_t$ are observed covariates on which the rate may depend

and evolves as a linear dynamical system.

$p(\beta_t|\beta_{t-1}) \sim \mathcal{N}(A\cdot \beta,Q)$

I'm trying to compare this model with a simpler model with fixed parameters (i.e. just a simple poisson regression). It is my understanding that to compare these models I need to calculate $p(y|\theta)$ in both cases but am unsure how to calculate this for the latent variable model.

In order to fit the latent model I've used the EM algorithm outlined here, making a laplace approximation to adjust for the nonlinear observation process: http://mlg.eng.cam.ac.uk/zoubin/course04/tr-96-2.pdf.

The authors provide a closed form expression of the joint likelihood $p(x,y)$ of data and latent variables but I'm unsure how to calculate the likelihood.

My current thinking is that I need to integrate out the latent variables $x$; i.e.

$p(y|\theta) = \int p(x,y|\theta) \ dx $

The only way I can think of doing this, since there is no closed form equation for the $p(x|y)$ is some kind of (markov chain) monte carlo approach. Is this correct or am I just way off?

  • $\begingroup$ How does $z_t$ relate to $\beta_t$ and $x_t$? $\endgroup$
    – jbowman
    Apr 10, 2019 at 18:58
  • $\begingroup$ Clarified, z is the rate of the poisson process: 𝑧=𝑒π‘₯𝑝(𝛽𝑑⋅π‘₯𝑑) $\endgroup$ Apr 10, 2019 at 19:03

1 Answer 1


Although not exact, the laplace approximation can be used to estimate the marginal likelihood.

$log \ p(Y|\theta) \approx log \ p(Y|\hat{\beta_{\theta}},\theta) + log \ p(\hat{\beta_{\theta}}|\theta) - \frac{1}{2} \cdot log \ | -H_{\theta} | + K$

where K is constant independent of $\theta$, $\hat{\beta_{\theta}}$ is the MAP estimate of $\beta$ condition on $\theta$.

see pp117 of http://www.columbia.edu/~kr2248/publication/Paninski10.pdf for details.

In general, it does seem monte carlo methods are required to calculate the exact marginal likelihood.


Your Answer

By clicking β€œPost Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.