2
$\begingroup$

I used to apply supervised machine learning for maximum few dozen "normal", natural features like human interpretable ones in Boston House Prices table. I usually try to understand each of them, think about how to preprocess them by sometimes binning continuous ones, re-categorizing and encoding categorical ones etc.

Now, however, I confront with very numerous (several hundred) features where this intimate one by one overview is clearly impossible. The easiest solution would be to accept the features as they are and handle them blindly en masse but in this way some evident optimization methods or even more importantly, some decisive corrections cannot be pointed out. For instance, R and/or Python Pandas are sometimes incorrectly identify numerical/categorical columns which is quite misleading if not spotted (missing data imputation must be applied very differently, and standardization is senseless in case of categorical ones, just mentioning two problems).

So my question is what is an appropriate way to handle and clear/correct these numerous features when one by one examination is out of question? Am I limited to use tree-based model types which are insensitive to NAs and scaling so no need for preprocessing?

(Just for being clear: this question is not about reducing the dimensionality.)

Improve this question
$\endgroup$
1
  • 1
    $\begingroup$ Boruta (I prefer R to python for this) with its RF that accounts for categorical mis-weights, doesn't care about multi-colinearity, and accounts for both nonlinearity and conditionality of functions is a decent first sieve for determining variable value. I have used it with 20k+ columns with good results. cran Boruta The h2o.ai tool also has good feature-reduction tools. $\endgroup$
    – EngrStudent
    Commented Feb 9 at 16:00

1 Answer 1

Reset to default
0
$\begingroup$

I would argue that you most certainly are not limited to just tree-based models. You could always try and drop samples with missing feature values, although of course this could lead to too many samples being lost.

You already mention imputation and correctly assess that different data-types would need different strategies. You could write a pre-processing script that for each feature checks the data type and then applies your chosen strategy. This would be pretty similar to your one-by-one analysis conceptually, but if your script is written correctly you can analyze a dataset of a few thousand features in size with ease.

Hope this helps!

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.