4
$\begingroup$

The model of a HMM consists of a latent Markov chain with state $X$ and transition probabilities $P(X^t \mid X^{t-1})$, and observation variables $Y$ that depend on the current latent state via $P(Y^t \mid X^t)$.

This results in the following causal graphical model:

image of HMM as graphical model

Question

What are reasons for why the arrows go from the latent state to the observed variable, and not the other way round?

I'm looking both for technical and philosophical reasons.

  • An example of a technical reason could be: This way, the model has fewer parameters.
  • Philosophical reasons could be based on the relationship between the causal structure of the model and the causal structure of reality. Shall the latent variable cause the observation, or shall they influence the latent state?

I'd also be very glad about pointers to literature that discusses this issue.

$\endgroup$
3
  • $\begingroup$ You're not going to find a philosophical reason related to causality because the HMM does not involve any; the fact that the observation distribution is specified conditional on the state does not imply causality. In fact, one of the main things people usually want to do is compute the reverse distribution (states conditional on observations). $\endgroup$
    – Chris Haug
    Commented Apr 13, 2017 at 12:14
  • $\begingroup$ @ChrisHaug But the variables (observations, hidden state) do correspond to aspects/variables of the real world. And between correlated variables, there should be a causal relationship according to Reichenberg's principle of common cause. Now you can use any model you like to represent the distribution of these variables. But maybe the model fits better if the arrows go along the true causal direction. Otherwise you might end up with a model that contains too many, or too few conditional independencies, as compared to the true distribution.. $\endgroup$
    – ziggystar
    Commented Apr 13, 2017 at 12:31
  • $\begingroup$ It is not at all required for the hidden state to correspond to anything "real" and no causality is assumed to exist in any direction. The HMM and state space models in general are a modelling device for "pushing" the time dependence away from the observations; it makes them conditionally independent. This simplifies computation. $\endgroup$
    – Chris Haug
    Commented Apr 13, 2017 at 13:28

1 Answer 1

1
$\begingroup$

Intuitively, the direction of the arrow makes sense because usually one interprets the observed part of the HMM as "symbols emitted by the hidden states".

However, it's worth noticing that bayesian networks are not necessarily causal graphs. Bayesian networks per se are simply one form of probabilistic graphical models, which graphically encode conditional independencies, which allows you to factorize the joint distribution.

So in this case the main goal of the bayesian network is to encode the fact that $Y_t$ is independent of $X_{t-1}$ given $X_t$. Or that $X_t$ is independent of $X_{t-2}$ given $X_{t-1}$ and so on.

To make the point clear, it's also very common to represent a Hidden Markov Model with another form of graphical model --- Markov Random Fields, and in this case the edges are undirected, as shown below:

enter image description here

In sum, the main point of the graph in the case of HMM is to encode the conditional independencies, not usually associated with causal inference per se. Of course, if you want, you could interpret them causally if that makes sense on the context of your application.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.