0
$\begingroup$

I have a n-dimensional data that maps to a 1D value.

I want to train a regressor, so given a new sample I can predict the outcome.

The problem is that I don't know which of this n-dimensions are good predictors and which ones are bad.

My approach is to first train N regressors, taking each input dimension alone and regressing it to obtain the target value, then I can see the r_squared value of each regressor to determine which dimensions are good predictors and which ones aren't.

The problem that I see, is that when taken alone they might be bad predictors but together they might perform better.

But if I take all of them, I just end with a single r_squared value telling me how the nD->1D regressor works.

Is there a metric that allows me to score how good is each dimension to I can apply a threshold and drop the ones that are not good?

Thanks

$\endgroup$
  • $\begingroup$ In a worst-case scenario, you could run separate regressions with every possible combination of regressors, and find which combination results in a model with the highest adjusted $R^2$. This may take more time but it should point you in the right direction. You may also want to consider out-of-sample testing to understand how well each model can predict, to avoid overfitting. $\endgroup$ – ERT Aug 2 '18 at 16:11
0
$\begingroup$

Firstly you should calculate correlation between your n-dimension data and your label column. That would give you knowledge which column you should consider to add to your model. Sometimes algorithm could not manage well on the multiple dimention and it's good to reduce it (It's called Curse of dimensionality). The possible tricks to cut number of dimentions : PCA, LDA, t-SNE.

You maybe should consider using ensemble algorithms (e.g bagging, boosting) which compute error and they try to reduce it on each iteration.

$\endgroup$
  • $\begingroup$ By using the pairwise correlations you would possibly drop the features that are important but only when considered together with other features... $\endgroup$ – Tim Aug 2 '18 at 10:46
  • $\begingroup$ But that's what I'm actually doing right? Getting the correlation between each individual dimension and the label. PCA gives you the variance in each direction and new coordinate system based on that variance, but a grater variance doesn't mean better correlation, so PCA will not give me the best correlated dimensions. $\endgroup$ – Sembei Norimaki Aug 2 '18 at 11:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.