0
$\begingroup$

I have the below sample code in which I am using the sklearn(scikit-learn=0.16.1) 20newsgroup dataset:

import pandas as pd
import numpy as np
from sklearn.datasets import fetch_20newsgroups
from sklearn.linear_model import LogisticRegression
from sklearn.multiclass import OneVsRestClassifier
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.classification import precision_recall_fscore_support
from sklearn import metrics

newsgroups_train = fetch_20newsgroups(subset='train')
newsgroups_test = fetch_20newsgroups(subset='test')
X_train = newsgroups_train.data
X_test = newsgroups_test.data
y_train = list(newsgroups_train.target)
y_test = list(newsgroups_test.target)
# y_train = [[x] for x in y_train]
# y_test = [[x] for x in y_test]

vectorizer = TfidfVectorizer(ngram_range = (1,1))
X_train = vectorizer.fit_transform(X_train)
X_test = vectorizer.transform(X_test)
clf = OneVsRestClassifier(LogisticRegression(C=(6), penalty='l1')).fit(X_train, y_train)
pred = clf.predict(X_test)
overall_precision,overall_recall,overall_fscore,overall_support = precision_recall_fscore_support(y_test,pred,average='weighted')
print "Precision " + str(overall_precision) + " " + "Recall " + str(overall_recall)

The output obtained is Precision 0.809072982209 Recall 0.806691449814. Now when I uncomment the comments in the above code and convert y_train and y_test from list to list of lists, the output changes to Precision 0.881659860014 Recall 0.631439192777. I understand that by doing it, I am claiming it to be a MultiLabel problem but I do not understand why that should change the precision or recall. My questions:

  1. Why is there a change of precision and recall?
  2. Which of the both are correct?
  3. How do I get a simple single data structure to use for y_train and y_test which can handle both MultiLabel and Single Labelled dataset.
$\endgroup$
0
$\begingroup$

I understood it after sometime. The version of sklearn I was using was 0.16. In that, a list will be taken as a multiclass classification and the other will be taken as a multilabel multiclass classification. The precision and recall are calculated in a different method for both these cases. In multiclass classification, the distance of the test case with each class is obtained and the least distance class is assigned. For multilabel multiclass classification, threshold is considered. As in, all those classes with threshold above 0.5 are given out leading to varying values in this case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.