# Hidden states that have never observed in the dataset!

I have been working on a dataset that has 5 discrete predictors and 1 binary response variable. Here is how to data looks like:

> head(train)
Y X1 X2 X3 X4 X5
1 2  1  3  4  6  1
2 2  1  3  2  6  1
3 2  1  3 10  6  1
4 2  1  3  4  6  1
5 2  1  3  7  6  3
6 2  1  3  2  6  1


$X_1$, $X_2$, $X_3$, $X_4$, and $X_5$ can take 2, 5, 16, 2, and 5 different discrete values, respectively. The dataset has 1327 rows, as I split it into train (70%) and test (30%) datasets:

> nrow(train)
[1] 929
> nrow(test)
[1] 398


Now, I want to apply HMM to predict $Y$, given $X=[X_1, X_2, X_3, X_4, X_5]$. I believe that updated emission transition matrix will provide me what I want. Here is what I have in my mind:

1. $Y$ is the observation sequence that can take 2 values.

2. Cartesian of $X$ which yields $2\times5\times16\times2\times5=1600$ unique hidden states.

3. Create HMM with initHMM(), and then apply baumWelch() to get updated $\theta=(\pi, A, B)$.

However, I have a couple of concerns.

1. Can I use data frequencies to get initial information for $\theta=(\pi, A, B)$? How will this better over just having a random start?
2. In the dataset, only 82 out of 1600 hidden states are observed. So, how could HMM make use of the rest 1518 states? Besides, computation time incredibly increases with 1600 states, may not even possible.

Thank you.

• As far as I understand, you do not need an HMM but a logistic regression with 5 covariates (independent variable). The HMM is a model for time series and the hidden state are something tha generates the Y and, possibly also the X... – niandra82 May 3 '16 at 15:54
• Actually, I already applied logistic regression, and now want to see performance of HMM. Is what you saying that I cannot apply HMM (if so why not?) or logistic regression fits better? – alamaranka May 3 '16 at 16:48
• because it is a model for time series, or at least for ordered variables since the dynamics of the latent states is a Markov process, i.e. is $s_t$ is the latent states you must be able to write the model in terms of $s_t|s-{t-1}$, for $t=1,2,....$ – niandra82 May 3 '16 at 16:49
• Actually, I am quite confused. Suppose I can use HMM for this dataset. What would be the hidden states, and observations? I would say $Y$ is observation and $X$ will be hidden states. Thank you for the comments. – alamaranka May 4 '16 at 0:03
• Hidden means that cannot be observed, then neither Y not X, but a new variable Z. The HMM is a particular mixture model and the latent variable (or hidden variable) is a discrete variable that represent the mixture membership, i.e. if $Z_i=k$, the i-th observation belongs to the i-th component of the misture – niandra82 May 4 '16 at 5:53