# Determine which dimensions are good predictors for regression

I have a n-dimensional data that maps to a 1D value.

I want to train a regressor, so given a new sample I can predict the outcome.

The problem is that I don't know which of this n-dimensions are good predictors and which ones are bad.

My approach is to first train N regressors, taking each input dimension alone and regressing it to obtain the target value, then I can see the r_squared value of each regressor to determine which dimensions are good predictors and which ones aren't.

The problem that I see, is that when taken alone they might be bad predictors but together they might perform better.

But if I take all of them, I just end with a single r_squared value telling me how the nD->1D regressor works.

Is there a metric that allows me to score how good is each dimension to I can apply a threshold and drop the ones that are not good?

Thanks

• In a worst-case scenario, you could run separate regressions with every possible combination of regressors, and find which combination results in a model with the highest adjusted $R^2$. This may take more time but it should point you in the right direction. You may also want to consider out-of-sample testing to understand how well each model can predict, to avoid overfitting. – ERT Aug 2 '18 at 16:11