1
$\begingroup$

I am doing a logistic regression to predict the outcome of a binary variable, say whether a journal paper gets accepted or not. The independent variable or predictors are all the phrases used in these papers - (unigrams, bigrams, trigrams). One of these phrases has a skewed presence in the 'accepted' class. Including this phrase gives me a classifier with a very high accuracy (more than 90%), while removing this phrase results in accuracy dropping to about 70%.

My more general (naive) machine learning question is:

  • Is it advisable to remove such skewed features when doing classification?
  • How do you such handle features which are intending to predict only one class?
  • Is there a method to check skewed presence for every feature and then decide whether to keep it in the model or not?
$\endgroup$
2
  • 1
    $\begingroup$ I'm surprised to see such a jump in accuracy. How many elements do you have in your training set, and do you compute the accuracy over a testing set that does not include any elements of the training set? $\endgroup$ – Franck Dernoncourt Nov 17 '13 at 18:10
  • $\begingroup$ related: stats.stackexchange.com/questions/10346/…. Btw, what's the phrase (just curious) ? $\endgroup$ – steffen Nov 18 '13 at 17:43
0
$\begingroup$
  • Is it advisable to remove such skewed features when doing classification?

    • In general, no. In various specific cases - yes. (If you hand engineered features, or used something very specific for example). Sometimes a strong feature is just that. (While in other cases it can be overfitting). What's the phrase? It might be "Accepted" or "published" for example :).
  • How do you such handle features which are intending to predict only one class?

    • Be VERY careful with your "negative set", assuming you're using discriminative classification. (And not 1 class or unsupervised methods). How large and "random"/diverse is your data? (Sources, fields, journals, types? ) Are oyu looking at the final text in the journal or the author submission? (See my question on how whether the feature is something like "accepted", "published", etc' ).
  • Is there a method to check skewed presence for every feature and then decide whether to keep it in the model or not?

    • Best way would be Chi square (plenty of inbuilt methods. eg, sci kit learn), or Mutual Information (entropy, etc'. Maybe mrmr?), or P. correlation.
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.