# Hidden Markov Model and Naive Bayes similarity

I understand Naive Bayes classifier and already have made a few implementaions.

What I dont understand is, considering that I have a training dataset with all the X observations and Y states, what stops me from plugging in the previous state($$Y_{n-1}$$) as a feature($$X_n$$) on the Naive Bayes? Would that transform it into a hidden markov chain?

I'm pretty sure the answer is no, but I can't understand why.

Hidden Markov Model assumes a relationship between $y_n$ and $y_{n+1}$.

For example say we are doing natural language processing, and $y_n$ denotes the $n$-th world in a sentence. If we know $y_n$ is "stack" then the probability of $y_{n+1}$ being "overflow" might be higher than knowing $y_n$ being something else say "cat".

While Naive Bayes does not make that assumption, instead it assumes that the observation sequence is i.i.d. Its more like $y$ is a random word from a random sentence, then knowing $y_n$ does not affect $y_{n+1}$.

Moreover, "plugging in the previous state(Y-1) as a feature(Xn) on the Naive Bayes" would make it a "reversed" Markov chain, as the arrow is now from $y_n$ to $y_{n-1}$. It assumes the same relationship if, in that natural language processing case, you read from right to left.

• Thanks for your answer. It now makes some sense! I guess i wasnt making a leap to the reverse markov chain. There are some cases where a reverse markov chain makes sense right? Also, when you have all the X and Y in a database does it make sense to still use a hidden markov model? May 22, 2016 at 2:39
• @user1020071 Glad I could help. yes the "reversed" Markov chains will make sense in some cases, but I think they are less commonly used than the ordinary ones, actually I just made up that name to show it's different :P HMMs are better used for sequential data, so if there's no order in the Xs and Ys, using HMM would make little sense. May 22, 2016 at 3:49
• It makes sense when the State, which is yet to be observed is not independent from last observed one, right? Just so i know i am thinking this straight May 22, 2016 at 22:20
• @MyIQisSub140HaveMercy right~ May 23, 2016 at 1:50
• This answer is actually pretty good. One year later, makes much more sense. Thank you Apr 13, 2017 at 3:23