Variance Explained by a Large Number of (Colinear) Variables

Question

Currently working on a project that explores how collectively, 121 variables about the environment, predict a single outcome. We run into two major issues:

1. Our variables are highly colinear. Rainfall tends to correlate with temperature which correlates with resources in the environment.
2. A lot of missing data. Our data is collected at the country-level, so Country A may have all 122 variables, but Country B may only have 50.

Sample size considerations aside (though I understand we are pushing them), how might we figure out what percentage of the variance in our single outcome is predicted by our 121 variables? In other words, what's a procedure we could use to get an R^2.

We have tried multiple-regression (doesn't address collinearity) and CART (too many missing values), though perhaps incorrectly.

Here is some sample data

| Country| Enviro.A | Enviro.B | Enviro.C | Enviro.D | Enviro.E | Outcome  |
| -------| -------- | -------- | -------- | -------- | -------- | -------- |
| A      | .63      | 1.33     | 5.84     | NA       | NA       | 3.98     |
| B      | .79      | 1.30     | 1.51     | NA       | 2.51     | 4.01     |
| C      | .77      | 1.04     | 4.34     | NA       | 1.87     | 4.21     |
| D      | .83      | .72      | 1.65     | NA       | NA       | 4.27     |
| E      | .83      | .97      | 4.50     | 1.09     | 2.00     | 4.12     |

Any feedback, thoughts, or references are incredibly appreciated. ~ A

Answer 1

Sample size considerations aside, the $R^2$ from a multiple regression will tell you the variance of the outcome explained by your 121 predictors. Collinearity is not a concern if you are interested in $R^2$ or making predictions. Those 121 predictor variables explain $X \%$ of your outcome whether or not the predictors are related to each other.

Collinearity becomes a problem if you are interested in the individual effects of the predictors in the multiple regression. The standard errors of the 121 effects may be inflated due to the collinearity, making inference difficult (wider confidence intervals).

Answer 2

Missing data are one big problem here. A multiple regression model using all of the predictors shown in your sample data would only include Country E, as statistical software does analysis on complete cases (rows with no missing data on the included predictors). Countries A through D lack at least one of the 5 Enviro. predictors and would thus be ignored in multiple regression.

Stef van Buuren's book is a great source of information on how to handle missing data. There's a lot to digest, but you need to see how the principles of dealing with missing data apply in your case.

Although collinearity per se isn't always an issue, as another answer rightly noted, you do have a related problem here. You are evaluating 121 predictor variables on a per-Country basis while there are fewer than 200 Countries on Earth, each with evidently only one Outcome. That low ratio of outcome values to predictors inherently leads to overfitting and unreliable modeling. Even if you overcome the missing data problem, you need to do something to deal with the case/predictor ratio.

Frank Harrell's course notes explain how to develop a coherent regression modeling strategy in circumstances with missing data and multiple correlated predictors. There are ways to identify redundant predictors or combine multiple related predictors into single predictors. Integrate that strategic approach with the nature of your data and the goal of your study.

Finally, this type of data often are present in time series that require special handling to deal with correlations over time. That doesn't seem to be the case here, but I leave that as a warning to others who might read this page.