1
$\begingroup$

I am trying to classify text documents using a huge corpora. Thats a huge tagset (more than 1000 tags). Corpus will have 1000 samples for each tag. But the tagset is not closed. New tags can be added in future as per the text content. There will be human intervention in between where I can get new tags if the situation arise. There can be multiple tags per text document.

nltk book talks about it in chapter 6. They referred to it as open-class classifiers in section 6.1 after the three bullet points.

Is it possible via some already implemented toolkit in python/ R/ C/ Java? If not I guess it is a problem of semi-supervised learning. I am familiar with nltk as well as scikit classifiers. Can those functions be used with other reinforcement learning toolkit?

I have no experience with reinforcement/ semi-supervised learning. I hope I am clear in the question. I will be thankful for your suggestions.

Thanks.

$\endgroup$
1
$\begingroup$

From your problem statement, it seems you are doing multiclass classification where number of classes is not known apriori along with human supervsion if needed. Mandrian forest and it's variant discussed in the paper Mandrian Forest can help you.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.