Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am trying to fit an HMM to a dataset resembling:

id0: A, A, A, C, B, A, C, B
id1: C, B, A, A, C, C
id2: B, A, A, C, B, B, A

In total there are three characters (A, B, C) and about 40k of such rows. I'm using R's HMMFit from the RHmm package.

As input to HMMFit one has to select the desired number of hidden states, so I can generate different models for different numbers of hidden states. In addition to the model, HMMFit reports the log-likelihood, the BIC criterium and the AIC criterium.

How does one evaluate the adequacy of such a model? Should one infer the model from training data as opposed to the entire dataset like I do now? If yes, how does one check if the trained model describes the test data?

Given different potentially adequate models generated for different numbers of hidden states, which one should I prefer?

share|improve this question
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.