I'm trying to use BIC/AIC for model comparison and want to know what the number of parameters is.

The models I'm unsure about are linear Gaussian state space models with nonlinear observations. Specifically I have latent variables $\beta_t$ that evolve over time as a linear dynamical system:

$p(\beta_t|\beta_{t-1}) \sim \mathcal{N}(A\cdot \beta,Q)$

where A is the dynamics matrix and Q is the covariance matrix of the innovation process.

The latent variables are then parameters of a Poisson observation model with rate $z = \beta_t \cdot x_t$ where $x_t$ are covariates:

$p(y_t|\beta_t;x_t) \sim \mathcal{Poiss}(z)$.

Im using an EM algorithm for inference (MCMC is not an option as datasets are too large), inferring $\beta$ in the E step and $A$ and $Q$ in the M-step.

Basically I'm entirely unsure what number of model parameters to use. Is it $ dim(A)^2 + dim(Q)^2$, do I factor in the latent variables? Any advice or places to read about this would be greatly appreciated! :)

  • $\begingroup$ What are your thoughts so far? What have you tried? Why do you think it isn't doing its job? There are things we know about A and Q from $\beta$ and $x_{t}$, what are they? $\endgroup$ Mar 29, 2019 at 10:20
  • $\begingroup$ My thoughts so far are to just use the likelihood function either on the entire dataset or on a held-out test-set, but this seems a bit weird with state-space models... I've tried to read around but the only thing I could find were some sources saying to use DIC for latent variable models. I haven't really tried anything because I wouldn't really know what to look for and deriving something myself, from first principles, is way beyond my mathematica expertise. I realise that conditional on a posterior distribution over $\beta$ we can infer A and Q but I'm not sure how thats relevant? $\endgroup$ Mar 29, 2019 at 10:32
  • 1
    $\begingroup$ I would use the dim of A and the dim of Q, assuming that Q is non-sparse. Some folks take only the diagonal and set all other values to zero, making the covariance into an axis-aligned hyper-ellipsoid, and reducing number of variables of Q to its square root. What do you know about the "dynamics" of A? I think A is square, and the same size as $\beta$ or $x_{i}$. $\endgroup$ Mar 29, 2019 at 10:38
  • $\begingroup$ yes Q is non-sparse and you are also right about the dimensions of A and Q. I'm not fully sure on the dynamics of A. From an initial look it seems the dynamics are not well approximated by a linear system... Fitting off-diagonal elements of A seems to result in awful overfitting, so I'm thinking of enforcing it to be diagonal. Do you know any sources for using dim(Q) and dim(A) as the number of parameters I could read more about? Thanks for your help! $\endgroup$ Mar 29, 2019 at 11:08


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.