3
$\begingroup$

I am fairly new in the area of text mining and want to practice my skills a little. I have the following task at hand which I want to work on. I have a large list of short texts (~100.000) and every text has on average 3 tags / labels assigned to it. What I want now is train a predictive model, with which I can assign labels to new unseen texts. The assumption about the labels is, all existing labels are included in my dataset.

I started with python sklearn and a basic tf.idf matrix representation combined with an one-vs.-rest support vector classification for each label, but the mean f1 score for that model are very low. Are there better ways of creating a predictive model for that kind of task?

As I mentioned before, this area is quite new to me so a little advice on how to tackle this problem would be more than welcome (e.g. some literature ?).

$\endgroup$
  • $\begingroup$ Excuse me, I am also very interested in this topic, so can you share your multi-labeled short text database? Thanks so much! $\endgroup$ – user43530 Apr 10 '14 at 8:45
  • 1
    $\begingroup$ A similar (?) problem was once a Kaggle competition problem, so check the forums to learn more what approaches were used by participants. $\endgroup$ – Tim Jan 3 '15 at 8:51
3
$\begingroup$

You'll want to familiarize yourself with multi-label classification, to better understand the problem you're working on. Tsoumakas et al is a good review to start with. Python scikit actually has multilabel classification functionality built in, so that might be a good out-of-the-box solution for you!

$\endgroup$
0
$\begingroup$

Make sure to remove stopwords and useless terms before training the model. I usually run such a classification model using Rapidminer and a single label, it works fine.

$\endgroup$
0
$\begingroup$

Take a look at Andrea's Esuli MP-Boost [LINK]. It is a multi-label variant of AdaBoost. It scales pretty well to large datasets and is quite efficient. The only downside perhaps is that you cannot restrict natively which label combinations are authorized (e.g. you may not want to label X and Z to be predicted at the same time for a short text). This might not be a problem for your application, though.

$\endgroup$
0
$\begingroup$

You can use SVM to build a one-vs-all classifier based on collection of labels that you can use to train the algorithm.

here is a couple of links that might help you.

http://www.sanjivk.com/pSVM_ICML13.pdf http://www.icmla-conference.org/icmla10/CFP_Tutorial_files/hakan.pdf

Cheers!

$\endgroup$
  • $\begingroup$ Could you add a few more details about your suggested approach to help the OP understand where this recommendation is coming from? $\endgroup$ – Matt Krause Sep 30 '15 at 0:08
0
$\begingroup$

If you feel like throwing Deep Learning at it might be a good idea, feel free to look at Magpie. Bear in mind though, that you need large labelled dataset in order for it to work well.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.