Calculating the degrees of freedom of a hidden Markov model (HMM)

Question

I am curious if there is a straightforward explanation for calculating the degrees of freedom of a hidden Markov model (HMM).

For example, take a simple HMM with a 1st-order Markov chain and 2 hidden behavioural states. The HMM is used to model the behavioural states giving rise to the movement patterns of a person walking on a hike. The 2 states are walking and resting, and can be described by the person's step lengths.

Thus, walking is characterised by medium to long step lengths and resting is characterised by short step lengths.

There are 1000 observations of the person walking (i.e. 1000 observations of step length).

Each state (walking and resting) has a gamma distribution (hence each state requires the estimation of a mean and standard deviation).

How would I calculate the degrees of freedom of such an HMM?

Answer 1

Hidden Markov models have two parts.

1. The hidden transition matrix: Describes the transition probabilities between the hidden states.
2. The emission matrix: Describes the probability of a particular observed state given occupation of a particular hidden state.

Typically HMMs are discrete, so your observations would be binned in to discrete ranges. You would then train your HMM on the discrete data and look at the two resulting matrices.

In your example you would notice:

• When my model is in state 1 (resting), my emission matrix has a high probability of giving an observation of certain observed states. And similarly for state 2.
• So the degrees of freedom here are a 2xN matrix where N is the number of discrete observed states. Each of the 2 rows of length N effectively describe a discretized probability distribution.

However, there are continuous formulations which extend the typical HMM theory, see for example Kalman Filters or Extended Kalman Filters.

This question is also somewhat relevant and might be useful to look at.