3
$\begingroup$

I have a binary supervised classification problem with about 62 features, by eye about 30 of them could have reasonable discriminating power. Depending on the situation I have between 12,000 and 2,000 samples ( I consider a number of cases but the features are the same for all ).I am using sklearn and the MLP does not have a dedicated feature selection tool like decision trees do. My question is what is the recommended way to preform feature selection here? I have read in the sklearn documentation that LDA should not be performed in a binary classification problem and PCA is under the unsupervised methods on the sklearn website.

Does anyone have any experience with this that could suggest a method?

Edit: Added number of samples

$\endgroup$
1
$\begingroup$

Most probably, you do not need dimensionality reduction.

People do dimensionality reduction if the problem is intractable, but with 62 features, it is not the case.

People also sometimes reduce dimensionality because the number of observations is too small in comparison with the number of features. But if your sample is small, using neural network is a bad idea anyway - use logistic reression or SVM instead, as it is more robust.

Dimensionality reduction and feature selection are also sometimes done to make your model more stable. But you can stabilize it by adding regularization (parameter alpha in the MLPClassifier).

Dimensionality reduction and feature selection lead to loss of information which may be useful for classification. So if you don't have a very serious reason for this, do not use PCA or LDA fith MLP.

$\endgroup$
  • $\begingroup$ I edited my question for the number of observations I have, I currently set alpha doing a grid serach. I was asked to look into dimensionality reduction but maybe I am going a bit overkill here. $\endgroup$ – MaggaP Nov 9 '17 at 17:33
0
$\begingroup$

Well, you didn't say how many observations you have as @David pointed out.

Also it seems to me, you are a bit confused about supervised and unsupervised learning. The MLP you are using to build your classifier is indeed a supervised method. But feature selection does not have a "correct" answer. You can try to reduce the dimensionality with minimizing the loss of information hidden in the data, but that is still a problem for unsupervised method. How will the features look like? How many of them should there be? There isn't a single answer. PCA is indeed an unsupervised method, that is widely used for this and you can definitely try to apply it to your data. Also there are many ways to determine how many components should you retain. Check out the elbow diagram and proportion of variance explained for example.

Also it should be pointed out that PCA is not a feature selection method, but rather a dimensionality reduction method. It doesn't select some features from the original dataset, but transforms it into new features that are "ranked" on how much they contribute to the information.

$\endgroup$
0
$\begingroup$

My suggestion to go with LIME framework https://github.com/marcotcr/lime

This tool will help you in analyzing feature importance.

It will help you in analyzing features on sample level and won't provide any information on model level.

$\endgroup$
  • $\begingroup$ What is this tool? How does it work to analyze feature importance? $\endgroup$ – gung - Reinstate Monica May 8 at 17:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.