In case the order of features can make a difference in the results of a classification approach, which classifier algorithms perform better? I know Naive Bayes/KNN use bag of words and ignore the order. Does SVM do the same (I understand it highly depends on its kernel)? And does feature selection approach do the same thing as the bag of words-based techniques?


1 Answer 1


Bag of words: it's a model to represent you text as a bag of words, which ignore the order of the original text (If you only include unigram words).

Naive Bayes/KNN/SVM are all machine learning classification approach by using the features(words) from bag of words model to predict. I think none of these methods take the order into consideration if you only include unigram words.

If you incorporate N-gram words (bigram, trigram..) into your bag of words model, the order of these features/words will be taken care of in all machine learning models

  • $\begingroup$ +1 The OP says "use bag of words and ignore order", when in fact its "are often used with bag of words, which ignores order". $\endgroup$
    – Wayne
    Commented Dec 30, 2017 at 0:28
  • $\begingroup$ Thanks. So if we consider n-grams we would have a bag-of-n_grams which still does not consider the order of the n-grams. The reason I am asking this follow-up question is that I am comparing SVM with a bag-of-n_grams and a traditional kernel with other text classification methods such as SVM with string kernels/word sequence kernels. $\endgroup$
    – Sarah
    Commented Dec 30, 2017 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.