A multi-label classification for tagging short text I am fairly new in the area of text mining and want to practice my skills a little. I have the following task at hand which I want to work on. I have a large list of short texts (~100.000) and every text has on average 3 tags / labels assigned to it. What I want now is train a predictive model, with which I can assign labels to new unseen texts. The assumption about the labels is, all existing labels are included in my dataset.
I started with python sklearn and a basic tf.idf matrix representation combined with an one-vs.-rest support vector classification for each label, but the mean f1 score for that model are very low. Are there better ways of creating a predictive model for that kind of task?
As I mentioned before, this area is quite new to me so a little advice on how to tackle this problem would be more than welcome (e.g. some literature ?). 
 A: You'll want to familiarize yourself with multi-label classification, to better understand the problem you're working on. Tsoumakas et al is a good review to start with. Python scikit actually has multilabel classification functionality built in, so that might be a good out-of-the-box solution for you!
A: Make sure to remove stopwords and useless terms before training the model. I usually run such a classification model using Rapidminer and a single label, it works fine.
A: Take a look at Andrea's Esuli MP-Boost [LINK]. It is a multi-label variant of AdaBoost. It scales pretty well to large datasets and is quite efficient. The only downside perhaps is that you cannot restrict natively which label combinations are authorized (e.g. you may not want to label X and Z to be predicted at the same time for a short text). This might not be a problem for your application, though.
A: You can use SVM to build a one-vs-all classifier based on collection of labels that you can use to train the algorithm.
here is a couple of links that might help you.
http://www.sanjivk.com/pSVM_ICML13.pdf
http://www.icmla-conference.org/icmla10/CFP_Tutorial_files/hakan.pdf
Cheers!
A: If you feel like throwing Deep Learning at it might be a good idea, feel free to look at Magpie. Bear in mind though, that you need large labelled dataset in order for it to work well.
