I want to start working on data-mining by parsing text. It seems like the best place to start is by processing n-grams out of text to try sentiment analysis.
Muffins are fine, I wouldn't say I like them though.
However, I'm curious to know if I should include punctuation or not. (I plan on starting with 3-grams and working up since I'm not sure 2-grams include enough information for accurate results.)
Muffins are fine | are fine [,] | I wouldn't say | ....
Since a "," was found, start over at next word after the ",". Instead of including the punctuation like normal.
Muffins are fine | are fine , | fine , I | , I wouldn't | ...
Can anyone tell me if this is a bad idea?