0
$\begingroup$

I am working with a pretty large dataset containing 760 rows and arround 58k-60k features and I'd like to perform a feature selection to reduce the dimensionality of those. After stardardising the data I've decide to try with the SelectKBest method from sklearn and I realized that I have to provide a number of features I want to select (named k).

Is there any way of tunning this k parameter? How can I know the proper amount of features to select out of the almost 60k that I have initially?

The dataset contains different cancer patients' genes expressions by the way. Each column is a different gene, that's why I have so many features.

Thanks!

Improve this question
$\endgroup$
1
  • $\begingroup$ Maybe you should look at some kind of regularization. Is likely those algorithms don't choose all important variables or choose wrong ones. Look at this danielezrajohnson.com/stepwise.pdf $\endgroup$
    – Allan
    Jul 26, 2022 at 13:11

1 Answer 1

Reset to default
1
$\begingroup$

Since you are using SelectKBest, I will assume you are using the data for a prediction problem. You could split your data into train-validation-test sets or use a train-test split with cross-validation on the training set. Then you can try different $k$ for feature selection and compare the performance on the validation set, for example by plotting accuracy against k. That should give you an idea about the trade-off between the number of features and predictive performance.

Improve this answer
$\endgroup$
1
  • 1
    $\begingroup$ I made more research about the problem I was trying to resolve and I saw that It was too easy for a ML model to solve it and at the end it didn't matter much how much features I select. But I'll take it into account for the future, thanks! $\endgroup$
    – Julen
    Aug 24, 2022 at 9:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.