Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am currently having a data set, class 1 with about 8000 short text files and class 2 with about 3000 short text files. I applied LibSVM and tried a couple of parameter combinations in the cross-validation experiment.

Generally the class 1 precision falls into the range of (85%, 90%); the class 2 precision falls into the range of (70% , 75%); the recall of both class 1 and class 2 fall into the range of (80% , 85%).

For the text classification purposes, I built text feature space following the common approaches, tokening the document, filtering the stopwords and building the word vector using tf-idf or binary frequency, etc. I also tried n-gram model to build the feature space. But these approaches did not improve the performance a lot. I would like to know are there any other ways that may help tune the LibSVM to improve the performance. LibSVM provides grid search for parameter setting up, but it runs pretty slow.

share|improve this question
add comment

3 Answers 3

up vote 1 down vote accepted

LibSVM hasn't been getting reliable performance for me, of late. Have you tried using [SVMLight] ever 1?

You might also try looking at which features are showing the most predictive power in your model, and adding some sort of enriched-type feature. For example, if I were classifying documents on whether they contain information related to protein-protein interaction, I wouldn't really care about the specific names of the proteins as predictive features. I would pre-process my documents and normalize all protein mentions with some common term that wouldn't normally occur in my documents, like "THISWORDUSEDTOBEAPROTEIN". Previous research (sorry, I can't think of any citations off the top of my head except my own paper Ambert & Cohen, 2012) has shown that this can lead to improved classifier performance by preventing the classifier from getting distracted by common genes (e.g., ADH1A) and ignoring rare ones, instead combining the predictive power of all genes into a single feature you could think of as "GeneMentioned".

share|improve this answer
add comment

If you haven't done so, you can usually gain efficiency by stemming the words, as I explain on this post.

Stemming will replace words by their stem, so that for instance 'sky' and 'skies', or 'hope', 'hopes' and 'hoped', will be recognized as identical. If your texts are in English, you will most likely end up using Porter's stemming algorithm of which you can find many implementations, one of the most famous being that included in NLTK (in Python). The post mentioned above contains a code snippet that shows how to use it.

Another idea that I develop in this post is to use co-inertia analysis (CIA), a forgotten variant of canonical analysis to reduce the dimension of the data, while keeping maximal correlation with your classes. You would end up with only two or three scores instead of the few hundreds that you must have in your corpus, which might give a better bias-variance deal.

share|improve this answer
add comment

A simple thing to try, if you haven't done so already, is to normalize each document vector so the magnitude is 1.

SVMs tend to perform better if the magnitude of each training vector is similar.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.