My objective is to classify sentences into useful (denote in boolean as 1) and not useful (denote in boolean as 0) categories.

I have about 525 features where 300 features are the most frequent and important keywords after removing stopwords and the rest are domain names.

The total number of documents I have is 793.

I manually labelled the classes as useful and not useful and I have about 93 useful and the rest (700) as not useful.

Below is the result of my logistic regression with the parameter values:

model = LogisticRegression(penalty='l1')
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.4, random_state=0)
model = model.fit(X, y)

enter image description here

Based on the results i got, it seems that the regression is doing a very bad job at classifying sentences as useful with really bad precision, recall and 1-score.

How can i improve the accuracy?

Note, as shown above, i already am implementing L1 regularization.

  • 1
    $\begingroup$ We don't have enough information to help. Could you provide a visualization of your points and respective classes? And if possible the boundary of your logistic regressions. That would be a start. $\endgroup$ – Ramalho Oct 8 '14 at 2:54
  • 1
    $\begingroup$ Your high precision on "non useful" and low precision on "usefull" makes me suspect that your model is optimistic in terms of labelling cases as "non useful". This might be a side effect of the difference of training examples for each class. My tip is for you to label/collect a more representative sample of "usefull" data or/and train the model with less "non usefull" examples. $\endgroup$ – Ramalho Oct 8 '14 at 3:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.