I have about 30000 book names assigned to 6 categories, and I want to build scalable and accurate classifiers. So far I have only been able to use Naive Baye's and LibLINEAR classifiers and they both give me an (almost) identical precision and recall values of 0.8 and 0.7 after 10 fold CV.
I am wondering if I would be able to do better if I were to use more complex models . The problem is that the time complexity of the sophisticated models seems to increase super-linearly with the number of training instances. SVM (SMO implementation from WEKA), for example, has been running for the past 3hrs already on this data, whereas the Naive Baye's and LibLINEAR finished in about 15mins and 40mins respectively.
I am trying to build a general framework for short texts classification (twitter, text messages etc.), and so will be running many experiments over varied data sets. I require techniques that scale and work well (don't we all :-)). Any suggestions?
Another question is with regard to dimension reduction. When I pre-process my text, I apply stemming, stopword removal and convert the text to tf-idf vector representation. Dimension reduction techniques (Info gain, in particular) again seems to be taking an inordinately long time. Any scalable way to do feature selection? Would pruning by tf-idf scores an acceptable approach?
Edit 1: By "Info Gain", I meant Information Gain . And currently I am not doing any feature selection.