Markov Property: $p({\bf x}_t | {\bf x}_1, \ldots, {\bf x}_{t-1}) = p({\bf x}_t | {\bf x}_{t-1})$

Consider the following model for which the hidden states are ${\bf x}_t$ and the observations are ${\bf y}_t$: \begin{align} {\bf x}_t &= A_1 {\bf x}_{t-1} + A_2 {\bf x}_{t-2} + \cdots + A_p {\bf x}_{t-p} + {\bf w}_t, && {\bf w}_t \overset{\text{iid}}{\sim} \mathcal{N}({\bf 0}, \Sigma_w) \tag{1} \\ {\bf y}_t &= B_0 {\bf x}_t + B_1 {\bf x}_{t-1} + \cdots + B_r {\bf x}_{t-r} + {\bf v}_t, && {\bf v}_t \overset{\text{iid}}{\sim} \mathcal{N}({\bf 0}, \Sigma_v) \tag{2} \end{align}

Assuming that $p<r$ and that all $A_i$ and $B_j$ are conformable, I can use block-matrix multiplication to give this model a state-space representation as follows:

\begin{align} \vec{{\bf x}}_t &= A \vec{{\bf x}}_{t-1} + \vec{{\bf w}}_t, && \vec{{\bf w}}_t \overset{\text{iid}}{\sim} \mathcal{N}(0, e_1' e_1 \otimes \Sigma_w) \tag{3} \\ {\bf y}_t &= B \vec{{\bf x}}_t + {\bf v}_t, && {\bf v}_t \overset{\text{iid}}{\sim} \mathcal{N}(0, \Sigma_v) \tag{4} \end{align} where $\vec{{\bf x}}_t = \begin{bmatrix} {\bf x}_t \\ {\bf x}_{t-1} \\ \vdots \\ {\bf x}_{t-r} \end{bmatrix}$, $A = \begin{bmatrix} A_1 & A_2 & \cdots & A_p & 0 &\cdots & 0 \\ I & 0 & \cdots & 0 & 0 &\cdots & 0 \\ 0 & I & \cdots & 0 & 0 & \cdots & 0 \\ \vdots & \vdots & & \vdots & \vdots & & \vdots \\ 0 & 0 & \cdots & I & 0 & \cdots & 0 \end{bmatrix}$, $\vec{{\bf w}}_t = \begin{bmatrix} {\bf w}_t \\ 0 \\ \vdots \\ 0 \end{bmatrix}$,

$B= \begin{bmatrix} B_0 & B_1 & \cdots & B_r \end{bmatrix},$ $e_1$ is a unit vector of conformable length, i.e. $e_1 = (1, 0, \ldots, 0)$, and $\otimes$ denotes the Kronecker product.

Typically, when you're working with a state-space model (SSM) you make the assumption that the Markov property holds. So some refer to it as the "Markov assumption."

The model above is not a state-space model, but as I have shown it can be given the state-space structure. So is the state-space structure sufficient for the Markov property to be true? In other words, is it true that $$p(\vec{{\bf x}}_t | \vec{{\bf x}}_1, \ldots, \vec{{\bf x}}_{t-1}) = p(\vec{{\bf x}}_t | \vec{{\bf x}}_{t-1})$$ because of the state-space structure?

In some papers it seems that merely having this structure is enough to say that the Markov property holds, e.g. Dynamic Hierarchical Factor Models (see page 7 and appendix A-1).

EDIT: Same question seems to be asked here: https://math.stackexchange.com/questions/3253996/is-it-correct-to-say-a-state-space-model-has-the-markov-property

Looking back at this question, the answer seems to be trivial. My mathematical training makes a clear difference between what is assumed to be true and what needs to be proven.

For SSMs (and for non-SSMs that can put into a state-space form), it seems that assuming the Markov property to be true (i.e. making the Markov assumption) is redundant because, as user @hejseb says, "it is precisely the conditional independence structure that you have." It's not an assumption you need to make because it's already true.

  • 1
    $\begingroup$ Yes, as you show that is precisely the conditional independence structure that you have. $\endgroup$ – hejseb Jun 13 at 18:31
  • $\begingroup$ I see. So it is not necessary to actually show that the LHS of $p(\vec{{\bf x}}_t | \vec{{\bf x}}_1, \ldots, \vec{{\bf x}}_{t-1}) = p(\vec{{\bf x}}_t | \vec{{\bf x}}_{t-1})$ actually equals the right? $\endgroup$ – user235125 Jun 14 at 21:18
  • $\begingroup$ I don't know why you need to show that. I think most people view it as a trivial matrix product. $\endgroup$ – hejseb Jun 16 at 12:09
  • 1
    $\begingroup$ @grxxvtony: yeah, I agree. To me, the state space formulation IMPLIES the markov property. It scares me a little though if books/references say that because I would think that, if you're doing state space modelling, you know a reasonable amount about it ??? Could we be incorrect ? I don't think so but there's always that possibility !!!!!! Always good to have an opened mind. Definitely looking forward to hearing if anyone disagrees-has a different viewpoint etc. $\endgroup$ – mlofton Jul 21 at 4:24
  • 1
    $\begingroup$ @grxxvtony: I think we are still correct but to be slightly picky, the property does depend on formulatiing the states correctly. The bottom of page 3 explains it better than I could. So, I would think of it as: If one chooses the states so that the state space model can be formulated, then it is markov. www-stat.wharton.upenn.edu/~stine/stat910/lectures/… $\endgroup$ – mlofton Jul 22 at 5:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy