Improve Accuracy of Model for Text Classification (sklearn)

Question

I want to classify (newspaper) texts to classes like sports, technology or politics (7 classes). The current classifier is Logistic Regression.

Therefore I've collect some features:

• Linguistic Features (19): like text length, avg. word length, sentence length, number of adjectives (divided by text length) etc.
• Word Features (1000): Number of occurrences of words, transformed with tf-idf (at the moment only uni-grams).
• Named Entities (100): By now we extract potential names of people and count them (divided by text length).

We end up with more than 1000 features. Applying our data to it, we end up with a accuracy between 10 and 20 percent what is near randomness.

So, next step was to scale the data (StandardScaler and MaxAbsScaler of sklearn), without centering the data, because it's a sparse matrix.

This improves the accuracy to round about 60 percent. Is this realistic that just scaling the data improves the accuracy dramatically?

Now what could be the next step to improve the accuracy? We tried out Dimension Reduction with SVD (because PCA does not seem to allow sparse data). We choose to reduce the dimensions to 100, but the accuracy got worse (59 percent).

Answer 1

This improves the accuracy to round about 60 percent. Is this realistic that just scaling the data improves the accuracy dramatically?

Linguistic Features may take values far larger than word features, and I expect word features to be much more important than linguistic features, so this may explain the performance boost you have obtained after scaling.

Now what could be the next step to improve the accuracy?

Convolutional neural networks often beat logistic regression for text classification. SVMs are also worth trying, which should be straightforward in sklearn. You should also try to increase the size of your n-grams when using LR or SVM.