Say I have 1000 potential predictors in a logistic regression. I don't have time to check each predictor manually for linearity. I could wait till after variable selection, but in that case I wonder if some predictors that have strongly non-linear relationship with the mean may not be selected because their non-linear relationship confounds their significance. suppose logit(P(Y=1))=B_0+B_1X_1+B_2g(X_2) for some unknown g, but I regress logit(P(Y=1))=b_0+b_1X_1+b_2X_2, could it be the case that B_2<>0 but b_2 is not significant? Prhaps because SE(b_2) is too large due to the poor assumption that g(x)= x? Or what if there is collinearity between X_1 and X_2 enough that b_2 appears not significant but there is less collinearity between X_1 and g(X_2)? If this happens I would not know that some of the rejected predictors are actually useful, and then I would be back to needing to find some way to check for appropriate transformations automatically. Is there a good way to automatically check for tranformations when there are hundreds of predictors?
This would be a good candidate for either a lasso or elastic net regression methodology, which perform variable selection as well as coefficient shrinkage for collinear variables. Include the main effect terms, plus quadratic (or higher power) terms to account for possible nonlinear relationships.
You can check out the maximal information coefficient statistic. It will compute the dependencies (both linear and nonlinear) pairwise among features and it can be very convenient for large numbers of features.
Here is a link to the website. http://www.exploredata.net/
I believe recently an R package called minerva was also released which computes this statistic for you.
Compute the eigenvalues and eigenvectors of X'X. Eigenvalues that are close to 0 will correspond to eigenvectors that represent a set of multicollinear columns.