3
$\begingroup$

I have a classification problem which is multi-label with N labels. I would like to know which method would be the better choice? Training N classifiers (1 for each label) or a single classifier which use K-hot encoding. I'm using a neural network for this with a softmax at the last layer. (Is there any chance that the softmax would cause a problem?) I'm using tensorflow for the model.

Improve this question
$\endgroup$

1 Answer 1

Reset to default
5
$\begingroup$

It mostly depends on your labels. Training one classifier with N labels (= multitask) makes more sense when labels are related. But one cannot predict in advance with 100% certainty whether multitask will outperform single task (= training N classifiers for N labels)

This paper studied the question: Caruana, R. (1997). Multitask learning: A knowledge-based source of inductive bias. Machine Learning, 28:41--75. doi:10.1023/A:1007379606734 ; http://link.springer.com/article/10.1023/A:1007379606734 ; http://sci-hub.cc/10.1023/A:1007379606734

Single task:

enter image description here

Multi task:

enter image description here

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.