As I am trying to automate the process of evaluating models for my prediction problem, I would like to verify the following concerning the process of creating a model consisting of multiple predictor variables for predicting a response variable.
Assume that polynomial regression is promising for solving our prediction problem.
The goal of feature selection is to select which predictor variables are useful and should be included in the model under construction. After having defined which factors are important, then we have to define what is the maximum polynomial degree for each of the "useful" predictor variables.
Does it mean that we have two ways for tuning the complexity of the model: (a) the number of features, and (b) the polynomial degree of each selected feature?
Do you think that the above process is sequential? Is it possible to separate the first step from the second?