scikit multi label classification

Question

I am trying to classify data into four different labels. The training data looks something like:

[
  [[0.1 , 0.6, 0.0, 0.3], 1, 10, 0, 0, 0],
  [[0.7 , 0.3, 0.0, 0.0], 0, 7, 22, 0, 0],
  [[0.0 , 0.0, 0.6, 0.4], 0, 0, 6, 0, 20],
  ...
]

Each row is an instance. First column contains the labels. Columns 2..n contain the features.

I am trying to understand how to train a multilabel classifier using SciKit OneVsRestClassifier but I just don't get how I must proceed in the code.

I have been able to do single-label classification for this dataset replacing the first column with a single value (eg. the first instance would map completely to the second label) but I would like the more nuanced multi-label output. This is what I use for the single label classifier (assuming above dataset):

labels = [1,0,3, ...]
data = [[1, 10, 0, 0, 0], [0, 7, 22, 0, 0], [0, 0, 6, 0, 20], ...]
clf = svm.SVC(kernel='poly')
clf.fit(data, labels)

Any idea how to convert this to multi-labels?

Thanks!

Answer 1

The Multi-label algorithm accepts a binary mask over multiple labels. So, for example, you could do something like this:

data = [
        [[0.1 , 0.6, 0.0, 0.3], 1, 10, 0, 0, 0],
        [[0.7 , 0.3, 0.0, 0.0], 0, 7, 22, 0, 0],
        [[0.0 , 0.0, 0.6, 0.4], 0, 0, 6, 0, 20],
        #...
       ]

X = np.array([d[1:] for d in data])
yvalues = np.array([d[0] for d in data])

# Create a binary array marking values as True or False
from sklearn.preprocessing import MultiLabelBinarizer
Y = MultiLabelBinarizer().fit_transform(yvalues)

clf = OneVsRestClassifier(SVC(kernel='poly'))
clf.fit(X, Y)
clf.predict(X)  # predict on a new X

The result for each prediction will be an array of 0s and 1s marking which class labels apply to each row input sample.

Given your data, though, I'm not sure this is what you want to do. For example, the third point has zero listed twice, which makes me think that you're not predicting multiple labels in an unordered OneVsRest manner, but actually predicting multiple ordered columns of labels: in that case, it might make sense to do a separate classification for each, e.g.

X = np.array([d[1:] for d in data])
Y = np.array([d[0] for d in data])
clfs = [SVC().fit(X, Y[:, i]) for i in range(Y.shape[1])]
Ypred = np.array([clf.predict(X) for clf in clfs]).T

With other classifiers, such as RandomForestClassifier, you can do this column-by-column prediction in one operation: e.g.

X = np.array([d[1:] for d in data])
Y = np.array([d[0] for d in data])
RandomForestClassifier().fit(X, Y).predict(X)

Of course, the array passed to predict should be on something different than the array passed to fit, but hopefully this makes the distinction clear.