8
$\begingroup$

I understand Naive Bayes classifier and already have made a few implementaions. enter image description here

What I dont understand is, considering that I have a training dataset with all the X observations and Y states, what stops me from plugging in the previous state($Y_{n-1}$) as a feature($X_n$) on the Naive Bayes? Would that transform it into a hidden markov chain?

I'm pretty sure the answer is no, but I can't understand why.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
6
$\begingroup$

Hidden Markov Model assumes a relationship between $y_n$ and $y_{n+1}$.

For example say we are doing natural language processing, and $y_n$ denotes the $n$-th world in a sentence. If we know $y_n$ is "stack" then the probability of $y_{n+1}$ being "overflow" might be higher than knowing $y_n$ being something else say "cat".

While Naive Bayes does not make that assumption, instead it assumes that the observation sequence is i.i.d. Its more like $y$ is a random word from a random sentence, then knowing $y_n$ does not affect $y_{n+1}$.

Moreover, "plugging in the previous state(Y-1) as a feature(Xn) on the Naive Bayes" would make it a "reversed" Markov chain, as the arrow is now from $y_n$ to $y_{n-1}$. It assumes the same relationship if, in that natural language processing case, you read from right to left.

Improve this answer
$\endgroup$
5
  • 1
    $\begingroup$ Thanks for your answer. It now makes some sense! I guess i wasnt making a leap to the reverse markov chain. There are some cases where a reverse markov chain makes sense right? Also, when you have all the X and Y in a database does it make sense to still use a hidden markov model? $\endgroup$
    – abriosi
    May 22, 2016 at 2:39
  • $\begingroup$ @user1020071 Glad I could help. yes the "reversed" Markov chains will make sense in some cases, but I think they are less commonly used than the ordinary ones, actually I just made up that name to show it's different :P HMMs are better used for sequential data, so if there's no order in the Xs and Ys, using HMM would make little sense. $\endgroup$
    – dontloo
    May 22, 2016 at 3:49
  • $\begingroup$ It makes sense when the State, which is yet to be observed is not independent from last observed one, right? Just so i know i am thinking this straight $\endgroup$
    – abriosi
    May 22, 2016 at 22:20
  • $\begingroup$ @MyIQisSub140HaveMercy right~ $\endgroup$
    – dontloo
    May 23, 2016 at 1:50
  • $\begingroup$ This answer is actually pretty good. One year later, makes much more sense. Thank you $\endgroup$
    – abriosi
    Apr 13, 2017 at 3:23

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.