7
$\begingroup$

Take a general linear Gaussian state space model (SSM)(aka Dynamic Linear Model DLM):

\begin{align} X_{t+1} &= FX_t + V_t \\ Y &= HX_t+W_t \\[10pt] V_t &\sim N(0,Q) \\ W_t &\sim N(0,R) \\ \end{align}

I am interested in the unidentifiability issues related to these models:

Hamilton (1994) states that ”in the absence of restrictions on F, H, Q and R, the parameters of the state-space representation are unidentified - more than one set of values for the parameters can give rise to the identical value of the likelihood function, and the data give us no guide for choosing among these”

Now I realise that this representation is not unique as multiplication by any orthonormal matrix $M$ produces a new representation:

\begin{align} MX_{t+1} &= MFM^{-1}M X_t + MV_t \\ Y &= HM^{-1}MX_t+W_t \end{align}

This type of unidentifiability where the observed values can be produced by various orthonormal transformations of the state variables is inherent to state space models.

However, I also came across another type of unidentifiability that seems to be related to the estimation method. In this case "Kalman Filtering." See the simple example starting on page 8 of this pdf.

In this case there is a linear transformation of the observation equation and an offsetting one made to the variance of the state equation

  1. Do both the transformations above give rise to the same kind of identifiability issues described by Hamilton (I believe they do but want to check)?

  2. Are there other ways in which identifiability issues can manifest themselves in Linear Gaussian SSMs?

  3. Is the fix always the same find constraints or analogously (Bayesian priors) that will ensure that the final parameters are correct?

Lastly this link in Matlab suggests that it is possible to build an "identifiable SSM". Unfortunately the link doesn't explain the theory. Thus:

  1. It is possible to translate any Linear Gaussian SSM into an "identifiable form"? Can someone please supply a link reference explaining how this works. It would seem at first blush that no matter what representation is used initially it would still be subject to the problems shown above?
$\endgroup$
  • $\begingroup$ There is no related example starting on page 8 - is this about the example on page 40? The state there is scalar, so it's just a special case of the "$M$ transformation" mentioned in OP (since for scalars $M\,F,M^{-1} = F$, only the variance changes in the state evolution). (Unclear if this is the answer to Q1 as it is unclear whether this is the one referred to). $\endgroup$ – Juho Kokkala Feb 21 '17 at 6:26
4
$\begingroup$

In my understanding you have to put restrictions on parameters, for example setting them to a constant, to ensure identification. There is no way to rewrite an unidentified model, while preserving all parameters, to a identified model.

There is however an algorithm to check whether a SS-model is identified. Try looking up the article:

J. Casals, A. Garcia-Hiernaux and M. Jerez, From general State-Space to VARMAX models, Mathematics and Computers in Simulation

In this article they give a cookbook to check for identification, but one step is left unexplained, it is the so-called "stair case algorithm" from the book,

H. H. Rosenbrock, State Space and Multivariable Theory, John Wiley, New York, 1970,

which I never had any luck locating.

$\endgroup$
  • 1
    $\begingroup$ Hi there, there is a related algorithm to the Rosebrock algorithm here: la.epfl.ch/files/content/sites/la/files/users/105941/public/… $\endgroup$ – Baz Jul 9 '15 at 15:12
  • $\begingroup$ The suggestion to place constraints on parameters in order to guarantee identifiability is a fairly obvious one. Is anyone perhaps aware of a concrete example of a set of constraints that would guarantee this for the linear dynamical system described in the original post? $\endgroup$ – ngiann Apr 16 '17 at 21:20
0
$\begingroup$

It is not true that Gaussian state space models (GSSM) are unindentifiable. First, inference on GSSM is inherently Bayesian. It can be shown that Kalman filter recurrences are identical to the equations used to update prior mean and covariances under a Bayesian perspective. Second, a sufficient condition for a GSSM to be identified is that its observability matrix is full rank. Take a look at chapter 5, page 143 of the book by West and Harrison.

$\endgroup$
  • $\begingroup$ Did you realize the question is about (non)identifiability of the parameters, $F,~H,~Q,~R$? Can you clarify in what sense the GSSM is identified if observability matrix is full rank? (I don't have the book and answers should anyway be self-sustainable) $\endgroup$ – Juho Kokkala Feb 21 '17 at 6:03
  • $\begingroup$ Ok, checked pages 143-144 from Springer's preview, don't see any connection with parameter identifiability $\endgroup$ – Juho Kokkala Feb 21 '17 at 6:16
  • $\begingroup$ In fact, I meant identifiability of states, which is the case one is interested in general. Matrices F and H are not in general fully estimated, in most cases it is possible to define them as functions of a smaller set of parameters which are identifiable. The same with covariances Q and R. Anyway, under a Bayesian perspective identifiability is not a concern, the question is if posterior distributions are proper or not. Remember that maximum likelihood is just Bayesian inference with uniformative priors. $\endgroup$ – Anselmo Feb 21 '17 at 15:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.