I understand Naive Bayes classifier and already have made a few implementaions. enter image description here

What I dont understand is, considering that I have a training dataset with all the X observations and Y states, what stops me from plugging in the previous state($Y_{n-1}$) as a feature($X_n$) on the Naive Bayes? Would that transform it into a hidden markov chain?

I'm pretty sure the answer is no, but I can't understand why.


Hidden Markov Model assumes a relationship between $y_n$ and $y_{n+1}$.

For example say we are doing natural language processing, and $y_n$ denotes the $n$-th world in a sentence. If we know $y_n$ is "stack" then the probability of $y_{n+1}$ being "overflow" might be higher than knowing $y_n$ being something else say "cat".

While Naive Bayes does not make that assumption, instead it assumes that the observation sequence is i.i.d. Its more like $y$ is a random word from a random sentence, then knowing $y_n$ does not affect $y_{n+1}$.

Moreover, "plugging in the previous state(Y-1) as a feature(Xn) on the Naive Bayes" would make it a "reversed" Markov chain, as the arrow is now from $y_n$ to $y_{n-1}$. It assumes the same relationship if, in that natural language processing case, you read from right to left.

| cite | improve this answer | |
  • 1
    $\begingroup$ Thanks for your answer. It now makes some sense! I guess i wasnt making a leap to the reverse markov chain. There are some cases where a reverse markov chain makes sense right? Also, when you have all the X and Y in a database does it make sense to still use a hidden markov model? $\endgroup$ – abriosi May 22 '16 at 2:39
  • $\begingroup$ @user1020071 Glad I could help. yes the "reversed" Markov chains will make sense in some cases, but I think they are less commonly used than the ordinary ones, actually I just made up that name to show it's different :P HMMs are better used for sequential data, so if there's no order in the Xs and Ys, using HMM would make little sense. $\endgroup$ – dontloo May 22 '16 at 3:49
  • $\begingroup$ It makes sense when the State, which is yet to be observed is not independent from last observed one, right? Just so i know i am thinking this straight $\endgroup$ – abriosi May 22 '16 at 22:20
  • $\begingroup$ @MyIQisSub140HaveMercy right~ $\endgroup$ – dontloo May 23 '16 at 1:50
  • $\begingroup$ This answer is actually pretty good. One year later, makes much more sense. Thank you $\endgroup$ – abriosi Apr 13 '17 at 3:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.