How to apply neural networks on multi-label classification problems?

Question

Description:

Let the problem domain be document classification where there exists a set of feature vectors, each belonging to 1 or more classes. For example, a document doc_1 might belong to Sports and English categories.

Question:

Using neural network for classification, what would the label be for a feature vector? would it be a vector constituting of all the classes such that 0 value is given to non-relevant classes and 1 for relevant classes? So if the class labels' list is [Sports, News, Action, English, Japanese], then for document doc_1 the label would be [1, 0, 0, 1, 0] ?

Answer 1

Yes, in multi-label learning the label information is often encoded as the binary vector you described. It is also easier so for evaluation.

We may want to check MULAN, an open source Java library for multi-label learning. It is a Weka extension and has implemented many multi-label classifiers, neural networks included. For example, you can find BP-MLL here.

Answer 2

This seems to be the paper you are looking for:

Min-Ling Zhang and Zhi-Hua Zhou: Multi-Label Neural Networks with Applications to Functional Genomics and Text Categorization

From the abstract:

In multi-label learning, each instance in the training set is associated with a set of labels, and the task is to output a label set whose size is unknown a priori for each unseen instance.In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e. Backpropagation for Multi-Label Learning, is proposed. ... Applications to two real- world multi-label learning problems, i.e. functional genomics and text categorization, show that the performance of BP-MLL is superior to those of some well-established multi-label learning algorithms.