I have 21 numerical features and I want to build a classification model. Having tried a bunch of classifiers to the original features I would like to look at feature engineering.
One approach would be to look at multivariate polynomials. Instead of looking at $X_1$ and $X_2$ I look at features $X_1,X_2,X_1X_2,X_1^2$ and $X_2^2$. As I have 21 features this quickly grows.
What are statistical approaches that help me to decide a good degree of my polynomials?
As a remark: SVMs with polynomial kernel and multivariate adaptive splines already try some polynomials - right? It would be interesting to produce them myself.