Removing collinear variables for LDA/QDA in R I'm looking for a function which can reduce the number of explanatory variables in my lda function (linear discriminant analysis).
Basically, I've loaded the dataset and ran the lda function on my binomial dependent variable explained by 30 independent variables but received a warning that the independent variables are collinear.
My professor has shown us stepwise feature selection (leaps package, regsubsets function) in a regression framework, but these codes aren't compatible for LDA/QDA.
 A: I do not know a standard procedure which could give you a single-call procedure to solve your problem. However, you can try to do that yourself. 
If two features are collinear it means that they should have a Pearson correlation coefficient far away from 0, and much closer to 1 or -1. Thus one way is to find those values for all your features.
library(corrplot)

x <- seq(0, 100, 1)
# colinear with x
y <- x + 2.3 
# almost colinear with x / some small gaussian noise 
z <- x + rnorm(mean = 0, sd = 5, n = 101)
 # uncorrrelated gaussian 
w <- rnorm(mean = 0, sd = 1, n = 101)

# this frame is made to exemplify the procedure
df <- data.frame(x = x, y = y, z = z, w = w)

corr.matrix <- cor(df)
corrplot.mixed(corr.matrix)

In order to give also some graphical sense to this process, I included corrplot package to draw some diagrams. A brief and consistent tutorial can be found here. Obviously, if you have a lot of features then a graphical plot would be the best way to look at this. 
Also, note that it is possible to have non-linear correlations. Your algorithm might be resistant to that or not. If not, you can search for other correlations like spearman's rho or kendall's tau. This can be computed in the same way using corr function with different parameters. See cor help page.
[Later edit]
As a threshold value I do not know if there is one established value. A good reason would be that it also depends on how the fitting procedure is able to handle that. I would try to experience with different values until I found a good one.
Also another procedure is to use variance inflation factor, but this requires more computation, although is a viable alternative. However I do not used it (which is not a plus, I am not a great expert). Searching for vif I found that there are also some automated precedures in R like the one presented here.
