2
$\begingroup$

Is anyone able to provide a simple explanation of how genetic algorithms can be used for feature selection in machine learning?

$\endgroup$
1
  • $\begingroup$ This is not necessarily the best criterion but it can be: top K features which are most prevalent in the best members of the N >> K independent generations which are an outcome of M genetic iterations. $\endgroup$
    – stans
    Sep 24 '19 at 15:20
1
$\begingroup$

A genetic algorithm maximizes (or sometimes minimizes) a target function value. In a feature selection task such value is, for one example, the relevance of a set of predictors to a target variable approximation. It is up to a researcher to decide what the relevance mean, I will give you a possible answer.

Suppose you want to do regression and your input space is too large (k >> n). You divide your sample into training and testing. Then by choosing a lot of different subsets of input features you run, for example, a random forest model on the training data, and get the loss function value for the testing sample. The later is your target function value, which you want to minimize.

Another example is linear regression. You may want to apply the genetic algorithm to select a feature subspace so that the F-statistic of the model is the highest. This time you maximize the F-statistic value. This, however, may not be the best practice for various statiastical reasons.

Just keep in mind that genetic algorithms try to solve a possible NP-hard problem by looking at input feature interactions rather then selecting individual features greedely and without accounting for the interactions with the other input features or their subsets.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.