Genetic Algorithms for Feature Selection Is anyone able to provide a simple explanation of how genetic algorithms can be used for feature selection in machine learning?
 A: A genetic algorithm maximizes (or sometimes minimizes) a target function value. In a feature selection task such value is, for one example, the relevance of a set of predictors to a target variable approximation. It is up to a researcher to decide what the relevance mean, I will give you a possible answer.
Suppose you want to do regression and your input space is too large (k >> n). You divide your sample into training and testing. Then by choosing a lot of different subsets of input features you run, for example, a random forest model on the training data, and get the loss function value for the testing sample. The later is your target function value, which you want to minimize.
Another example is linear regression. You may want to apply the genetic algorithm to select a feature subspace so that the F-statistic of the model is the highest. This time you maximize the F-statistic value. This, however, may not be the best practice for various statiastical reasons.
Just keep in mind that genetic algorithms try to solve a possible NP-hard problem by looking at input feature interactions rather then selecting individual features greedely and without accounting for the interactions with the other input features or their subsets.
