Markov chain prediction with unknown states

Question

I want to make a prediction of the next state based on a training and test dataset. So I split my data into a train and test set and calculate the MLE on the train set and want to predict the next state on the test dataset.

The problem now is, that there can be states in the test set, that never have been learned via the train set. How would I cope with that?

A similar problem occurs, if I have a similar train/test setup, and want to calculate the log-likelihood on the test dataset, with states never occuring in the train set.

Answer 1

You almost always have such problems with learned systems. One solution is to assume a prior distribution over your parameters, for example a Dirichlet distribution which would basically lead you to the assumption that you observed state x or transition y at least z times.

Answer 2

The simple solution is to stratify your data by states and then partition each state randomly into the training and testing sets. Of course, this might just push the problem from the testing stage to the deployment stage if the real world has states that your data does not.

The comment by kutschkem is a more general solution: place a prior distribution on your parameters.