# How to reduce dimensions of variable space w.r.t. single response variable? CCA?

My dataset is presence/absence (or relative abundance score) of 100 species on 5000 squares, and for each square I have ~100 environmental variables (many of them strongly correlated).

I want to reduce the dimension of variable space, i.e. reduce those 100 variables to only the most important dimensions (for further use in non-linear methods like GP etc.). First idea would be to use PCA or factor analysis, but these only consider the variable space itself; but I want the dimension reduction process to work with respect to the response variable (abundance of one particular species), i.e. I want it to consider how important the variables are for the response variable. Because in simple PCA (which would take only the environmental variables), ordination axes might seem to explain a lot of the variability of the environment itself, but if you look how these are actually important for explaining the variability of the species, perhaps the dimension might be reduced even much more.

The closest thing which came into my mind was CCA, because this is what CCA does with its ordination axes, right? I am able to run the analysis for all species together, but how to get the CCA for just one species? This is the error I get when I run the analysis with a single species:

require(vegan)
data(varespec)
data(varechem)
vare.cca <- cca(varespec[,1,drop=FALSE], varechem)
#Error in cca.default(varespec[, 1, drop = FALSE], varechem) :
#  all row sums must be >0 in the community data matrix


Questions:

1. Is it possible to run CCA on a single response variable? Does it make sense?
2. I know that the analysis wouldn't be multivariate in case of just a single species... perhaps there is a univariate equivalent for CCA?
3. Other solutions?
• You're thinking of multiple regression. Dec 2, 2019 at 17:30
• Why are you doing CCA? Sounds like you want factor analysis Dec 2, 2019 at 17:46
• @PeterFlom-ReinstateMonica 1) factor analysis, i.e. something like PCA, AFAIK only takes into account the environmental variables and their correlations; not the "response" variable, in this case abundance of 1 species (I updated the question in this regard). I want to reduce the dimensions of environmental variables not just per se, but with respect to the abundance of that species. Which is what I thought CCA does on multiple species, but I need it for one species. 2) I'm not insisting on CCA, it's just the only one which I know of, closest to my purpose. Dec 3, 2019 at 22:12
• what does 'with respect to' mean? could you be more specific? why do you want to do this? I'm struggling to see what you're getting at. Why is doing linear regression, which is mathematically equivalent to a univariate cca, not what you want? Dec 4, 2019 at 0:11
• @user257566 ok. Thanks for the comment. I've updated the question, hope it's clearer now? Feel free to ask! Doing linear regression is not what I need - I want to reduce the environmental variable space and then use the reduced dimensions into some other non-linear models like Gaussian Process. Dec 4, 2019 at 14:52

PLS regression (Partial Least Squares or Projection to Latent structures - both mean the same thing and underlying process), as suggested by user257566 is a very useful way to go for this type of query. Unlike PCA it uses external information to guide the act of data reduction, meaning that underlying data processes related to that external information are boosted compared to those that are not related (as compared to PCA).

For determining each latent variable PLS works by alternating between least squares optimisation of the X matrix and the Y matrix over several iterations, leading to components that attempt to simultaneously described the X and Y matrices as efficicently as they can. PLS theory

You mention in the comments to one question 'I've been looking into pls, but there are many different models (plsr, pcr, ...) and it's hard for me to see which one is the one'. PCR is nothing to do with PLS - PCR is principle components regression, where the data reduction is a plain PCA without reference to the Y data. For PLS, originally a distinction between Y vectors and matrices was made (PLS-1 and PLS-2) but the distinction was arbitrary and is now largely ignored. As with PCA many tweaks have been proposed over the years to create new versions of PLS, whether those are relevant depends on the assumptions associated with them and whether they apply to your data. Variable selection can be worked into the same data workflow if it would be useful to trim out less useful variables. Within PLS itself it is possible to use jack-knifing/ uncertainty testing (essentially throwing out variables that are poor predictors). Its possible to use a crude correlation or significance test between each variable and the response (if you are at an exploratory phase of analysis it would be better to use a lenient threshold so you don't throw out variables that are insignificant on their won but which combine with the other well to help the model)

Some regression methods explicitly use variable restriction as an alterantive way to avoid instability due to multiple collinearity. Ridge regression, LASSO and elastic net are popular. There is lots of info on these on CV and the assumptions underlying them that should help to determine if they are relevant. Ridge and Lasso

Formulas

Assumptions

PCR PLS Ridge and LASSO

• thanks for making my question on PLS clearer! Would you know how to get the transformation matrix from the R pls package? (The one which transforms the env. convariates to the component axes). Dec 9, 2019 at 22:13
• I've coded pls for myself in Matlab, but never used it in R I'm afraid. You do need to be sure what you are looking for - the PLS reduces the data into latent variables or components (similar to PCA, but reduced based on X and Y), then it uses these uncorrelated components in a regression, giving each a weight. this means you end up with a single vector of regression coefficients. The components will give you insight into all the independent processes contributing to the dataset, while the regression coefficients will give you the overall picture. Dec 10, 2019 at 11:06
• Here's the link for the package rdrr.io/cran/pls/man/coef.mvr.html . Based on that it seems that coef.mvr returns an array of regression coefficients. as far as I can tell loadings is the function for getting the components. Dec 10, 2019 at 11:11

I suggest using forward selection for variables, basing on p-values in logistic regression. That is, first we choose the first variable, that has the least p-value as a covariate in the univariate logistic regression model, where we take the presence of the species as the outcome. Then we add to the model another covariate with the least p-value, then the third covariate with the least p-value, and so on.

You may use also some measure of accuracy in cross validation or with a hold-out sample, instead of p-values.

Stepwise selection may also be appropriate. That is, at each step, you may remove covariates with p-values greater than some threshold.

If the distribution of the outcome is not binary, we are to consider linear regression. Or quantile regression if the distribution is severely skewed or has a big atom at zero.

• Hi Viktor, thanks for the answer! What you are advising is a classic regression, which I've already done and is not what I need (or at least, I don't really see how would it help me :-)) . I've updated the question based on other comments, so it's perhaps more clear now, hopefully. Feel free to ask! Dec 4, 2019 at 14:50