5
$\begingroup$

I'm new to concept of HMMs. I have trained 2 HMMs separately.

HMM1 is trained with symbols A, B, C.

HMM2 is trained with symbols D, E, F.

I have a set of observation symbols in the set V={A,B,C,D,E,F}.

In testing phase, I'm extracting a symbol by trying to associate a test vector to one of the symbols in V (using euclidean distance to cluster centers).

How would I determine the log likelihood of a Observation Sequence A, B, C,... if that HMM was only trained with the symbols D, E, F....?

Or is it that log likelihood isn't defined in this case?

Is it necessary that observation symbols have to be shared among various HMMs? i.e picked from a common observation Symbol set V?

Please help me figure out what I'm doing wrong?

Improve this question
$\endgroup$
  • $\begingroup$ have you tried looking at iohmm (input-output hmm) ? might be more suitable for this case. $\endgroup$ – Ran Feb 15 '12 at 7:24
6
$\begingroup$

This is a classic Black Swan problem. HMM1 will assign zero likelihood to symbols D, E, F and HMM2 will assign zero likelihood to symbols A, B, C. Essentially from HMM1's perspective, D, E, F are impossible, while from HMM2s perspective D, E, F are. They will never predict them. (Note that there is nothing about HMMs in this answer -- you could replace "HMM" with "classifier" or "model" and the previous statement would still hold.)

If you knew something about the relationship between the symbols A, B, C and D, E, F you could get creative with mapping them between each other.

In short, the loglikelihood of that sequence, i.e. a sequence A, B, C using a model trained on D, E, F is always -inf (= log 0).

Improve this answer
$\endgroup$
  • $\begingroup$ I figured that! But, is such a problem present in a real-time scenario? Like, a symbol/observation being unique to a HMM and not occur in other HMM? $\endgroup$ – garak Feb 14 '12 at 19:04
  • $\begingroup$ I can't think of a scenario where this would (have to) happen. Can you describe the context in which you encountered it? $\endgroup$ – Andrew Rosenberg Feb 14 '12 at 19:17
  • $\begingroup$ Thank you for your patience. Its kind of hard to describe it here as space is limited. How would I generate a test sequence/observations from a feature vector when I have a set of HMMs (HMM1, HMM2, etc) so that I would not encounter above problem? Do different HMMs have their own set of symbols(codebooks) or do they share same set of symbols(common codebook). In later case, some unique symbols per class/action are likely to occur. $\endgroup$ – garak Feb 15 '12 at 0:17
  • 2
    $\begingroup$ I am having a difficult time parsing your questions here. Please describe the context in which you can encounter the above problem. When you train a HMM, you train on an observation sequence, you expect that the test observation sequence to be made up of the same tokens. Different HMMs can definitely have their own sets of symbols, and the can have different symbols. I suspect, based on these followup questions, that you may need to revisit some fundamentals. $\endgroup$ – Andrew Rosenberg Feb 15 '12 at 3:54
1
$\begingroup$

Depending on how you define an observation, you can solve this problem by have a pseudo observation for rare training observations or unseen observations, e.g. number for all numbers. That way, when the HMM encounters an unseen observation, it looks for the closest pseudo observation. See 2.7.1 in this for more details.

On the other hand, if you can not have pseudo observation in you HMM model, the simplest way to handle unseen observation is just assign them zero probabilities!

Improve this answer
$\endgroup$
  • $\begingroup$ This is a great answer, but maybe you can adapt it a little bit to the question. You just gave the same answer twice. $\endgroup$ – Ferdi Oct 18 '17 at 20:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.