Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm new to concept of HMMs. I have trained 2 HMMs separately.

HMM1 is trained with symbols A, B, C.

HMM2 is trained with symbols D, E, F.

I have a set of observation symbols in the set V={A,B,C,D,E,F}.

In testing phase, I'm extracting a symbol by trying to associate a test vector to one of the symbols in V (using euclidean distance to cluster centers).

How would I determine the log likelihood of a Observation Sequence A, B, C,... if that HMM was only trained with the symbols D, E, F....?

Or is it that log likelihood isn't defined in this case?

Is it necessary that observation symbols have to be shared among various HMMs? i.e picked from a common observation Symbol set V?

Please help me figure out what I'm doing wrong?

share|improve this question
have you tried looking at iohmm (input-output hmm) ? might be more suitable for this case. – Ran Feb 15 '12 at 7:24
up vote 4 down vote accepted

This is a classic Black Swan problem. HMM1 will assign zero likelihood to symbols D, E, F and HMM2 will assign zero likelihood to symbols A, B, C. Essentially from HMM1's perspective, D, E, F are impossible, while from HMM2s perspective D, E, F are. They will never predict them. (Note that there is nothing about HMMs in this answer -- you could replace "HMM" with "classifier" or "model" and the previous statement would still hold.)

If you knew something about the relationship between the symbols A, B, C and D, E, F you could get creative with mapping them between each other.

In short, the loglikelihood of that sequence, i.e. a sequence A, B, C using a model trained on D, E, F is always -inf (= log 0).

share|improve this answer
I figured that! But, is such a problem present in a real-time scenario? Like, a symbol/observation being unique to a HMM and not occur in other HMM? – garak Feb 14 '12 at 19:04
I can't think of a scenario where this would (have to) happen. Can you describe the context in which you encountered it? – Andrew Rosenberg Feb 14 '12 at 19:17
Thank you for your patience. Its kind of hard to describe it here as space is limited. How would I generate a test sequence/observations from a feature vector when I have a set of HMMs (HMM1, HMM2, etc) so that I would not encounter above problem? Do different HMMs have their own set of symbols(codebooks) or do they share same set of symbols(common codebook). In later case, some unique symbols per class/action are likely to occur. – garak Feb 15 '12 at 0:17
I am having a difficult time parsing your questions here. Please describe the context in which you can encounter the above problem. When you train a HMM, you train on an observation sequence, you expect that the test observation sequence to be made up of the same tokens. Different HMMs can definitely have their own sets of symbols, and the can have different symbols. I suspect, based on these followup questions, that you may need to revisit some fundamentals. – Andrew Rosenberg Feb 15 '12 at 3:54

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.