I'm trying to classify messages into different categories using an SVM. I've compiled a list of desirable words/symbols from the training set.
For each vector, which represents a message, I set the corresponding row to
1 if the word is present:
"corpus" is: [mary, little, lamb, star, twinkle]
first message: "mary had a little lamb" -> [1 1 1 0 0]
second message: "twinkle little star" -> [0 1 0 1 1]
I think this is fairly common setup with SVM, but my question is, with thousands of words in the set, what if there's only 1-2 words per message that actually show up? Is the linear dependence of my set of training vectors going to adversely affect the ability of the algorithm to converge?