0
$\begingroup$

I am working on a dataset in which I have thousands of binary features and a binary response. From the interpretation side of things I would like to fit a SVM model combined with some sort of regularisation to reduce the number of variables while keeping a good AUC. Is there such thing on Python? It would really help me understand what features are better able to predict response since these are drugs that are classified as active (1) or inactive (0) against a protein target in cancer. Any other suggestion will be welcome.

Kind regards,

Improve this question
$\endgroup$
1
  • $\begingroup$ I think you should check the scikit-learn package. In this package, you can find an implementation of SVM with all the features you've mentioned: scikit-learn.org/stable/modules/svm.html $\endgroup$
    – bilibraker
    Jun 12, 2020 at 14:06

1 Answer 1

Reset to default
1
$\begingroup$

You can run SVM and, if you wish, different feature selection mechanisms, quite easily with scikit-learn.

Hint: Note that SVM is already regularized (margin is maximized = weight vector norm is minimized). So you can try first with vanilla linear SVM. Exactly because of its inherent regularization it is known to work well even when there are millions of features. In my experiences sometimes it works better like this than when performing feature selection in pre-processing.

Improve this answer
$\endgroup$
1
  • $\begingroup$ I found papers which combined feature preprocessing, random forest and SVM to perform feature selection. There is also a function in sklearn to perform recursive feature elimination together with cross-validation, did you ever use it? It calculates the importance of each coefficient (i think in terms of its absolute value), and prunes each feature at a time computing CV. I am tempted to f-score my features, run a bunch of models at different cut-offs and keep the best in terms of auc $\endgroup$
    – Ramon Soto Garcia
    Jun 12, 2020 at 20:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.