How to reduce dimensions of variable space w.r.t. single response variable? CCA?

Question

My dataset is presence/absence (or relative abundance score) of 100 species on 5000 squares, and for each square I have ~100 environmental variables (many of them strongly correlated).

I want to reduce the dimension of variable space, i.e. reduce those 100 variables to only the most important dimensions (for further use in non-linear methods like GP etc.). First idea would be to use PCA or factor analysis, but these only consider the variable space itself; but I want the dimension reduction process to work with respect to the response variable (abundance of one particular species), i.e. I want it to consider how important the variables are for the response variable. Because in simple PCA (which would take only the environmental variables), ordination axes might seem to explain a lot of the variability of the environment itself, but if you look how these are actually important for explaining the variability of the species, perhaps the dimension might be reduced even much more.

The closest thing which came into my mind was CCA, because this is what CCA does with its ordination axes, right? I am able to run the analysis for all species together, but how to get the CCA for just one species? This is the error I get when I run the analysis with a single species:

require(vegan)
data(varespec)
data(varechem)
vare.cca <- cca(varespec[,1,drop=FALSE], varechem)
#Error in cca.default(varespec[, 1, drop = FALSE], varechem) : 
#  all row sums must be >0 in the community data matrix

Questions:

1. Is it possible to run CCA on a single response variable? Does it make sense?
2. I know that the analysis wouldn't be multivariate in case of just a single species... perhaps there is a univariate equivalent for CCA?
3. Other solutions?

Answer 1

PLS regression (Partial Least Squares or Projection to Latent structures - both mean the same thing and underlying process), as suggested by user257566 is a very useful way to go for this type of query. Unlike PCA it uses external information to guide the act of data reduction, meaning that underlying data processes related to that external information are boosted compared to those that are not related (as compared to PCA).

For determining each latent variable PLS works by alternating between least squares optimisation of the X matrix and the Y matrix over several iterations, leading to components that attempt to simultaneously described the X and Y matrices as efficicently as they can. PLS theory

You mention in the comments to one question 'I've been looking into pls, but there are many different models (plsr, pcr, ...) and it's hard for me to see which one is the one'. PCR is nothing to do with PLS - PCR is principle components regression, where the data reduction is a plain PCA without reference to the Y data. For PLS, originally a distinction between Y vectors and matrices was made (PLS-1 and PLS-2) but the distinction was arbitrary and is now largely ignored. As with PCA many tweaks have been proposed over the years to create new versions of PLS, whether those are relevant depends on the assumptions associated with them and whether they apply to your data. Variable selection can be worked into the same data workflow if it would be useful to trim out less useful variables. Within PLS itself it is possible to use jack-knifing/ uncertainty testing (essentially throwing out variables that are poor predictors). Its possible to use a crude correlation or significance test between each variable and the response (if you are at an exploratory phase of analysis it would be better to use a lenient threshold so you don't throw out variables that are insignificant on their won but which combine with the other well to help the model)

Some regression methods explicitly use variable restriction as an alterantive way to avoid instability due to multiple collinearity. Ridge regression, LASSO and elastic net are popular. There is lots of info on these on CV and the assumptions underlying them that should help to determine if they are relevant. Ridge and Lasso

Formulas

Assumptions

PCR PLS Ridge and LASSO

Answer 2

I suggest using forward selection for variables, basing on p-values in logistic regression. That is, first we choose the first variable, that has the least p-value as a covariate in the univariate logistic regression model, where we take the presence of the species as the outcome. Then we add to the model another covariate with the least p-value, then the third covariate with the least p-value, and so on.

You may use also some measure of accuracy in cross validation or with a hold-out sample, instead of p-values.

Stepwise selection may also be appropriate. That is, at each step, you may remove covariates with p-values greater than some threshold.

If the distribution of the outcome is not binary, we are to consider linear regression. Or quantile regression if the distribution is severely skewed or has a big atom at zero.