I have a n-dimensional data that maps to a 1D value.
I want to train a regressor, so given a new sample I can predict the outcome.
The problem is that I don't know which of this n-dimensions are good predictors and which ones are bad.
My approach is to first train N regressors, taking each input dimension alone and regressing it to obtain the target value, then I can see the r_squared value of each regressor to determine which dimensions are good predictors and which ones aren't.
The problem that I see, is that when taken alone they might be bad predictors but together they might perform better.
But if I take all of them, I just end with a single r_squared value telling me how the nD->1D regressor works.
Is there a metric that allows me to score how good is each dimension to I can apply a threshold and drop the ones that are not good?