Im trying to calculate the likelihood function of my latent state space model.

My model has Poisson observations

$p(y_t|\beta_t;x_t) \sim \mathcal{Poiss}(z)$.

where $z$ is the rate of the poisson process, $z = exp(\beta_t \cdot x_t)$ and $x_t$ are observed covariates on which the rate may depend

and evolves as a linear dynamical system.

$p(\beta_t|\beta_{t-1}) \sim \mathcal{N}(A\cdot \beta,Q)$

I'm trying to compare this model with a simpler model with fixed parameters (i.e. just a simple poisson regression). It is my understanding that to compare these models I need to calculate $p(y|\theta)$ in both cases but am unsure how to calculate this for the latent variable model.

In order to fit the latent model I've used the EM algorithm outlined here, making a laplace approximation to adjust for the nonlinear observation process: http://mlg.eng.cam.ac.uk/zoubin/course04/tr-96-2.pdf.

The authors provide a closed form expression of the joint likelihood $p(x,y)$ of data and latent variables but I'm unsure how to calculate the likelihood.

My current thinking is that I need to integrate out the latent variables $x$; i.e.

$p(y|\theta) = \int p(x,y|\theta) \ dx $

The only way I can think of doing this, since there is no closed form equation for the $p(x|y)$ is some kind of (markov chain) monte carlo approach. Is this correct or am I just way off?

  • $\begingroup$ How does $z_t$ relate to $\beta_t$ and $x_t$? $\endgroup$ – jbowman Apr 10 '19 at 18:58
  • $\begingroup$ Clarified, z is the rate of the poisson process: 𝑧=𝑒𝑥𝑝(𝛽𝑡⋅𝑥𝑡) $\endgroup$ – user3235916 Apr 10 '19 at 19:03

Although not exact, the laplace approximation can be used to estimate the marginal likelihood.

$log \ p(Y|\theta) \approx log \ p(Y|\hat{\beta_{\theta}},\theta) + log \ p(\hat{\beta_{\theta}}|\theta) - \frac{1}{2} \cdot log \ | -H_{\theta} | + K$

where K is constant independent of $\theta$, $\hat{\beta_{\theta}}$ is the MAP estimate of $\beta$ condition on $\theta$.

see pp117 of http://www.columbia.edu/~kr2248/publication/Paninski10.pdf for details.

In general, it does seem monte carlo methods are required to calculate the exact marginal likelihood.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.