I'm learning by building models on UCI datasets and I've found interesting and difficult (for me ;-)) dataset

I'd like to build 6 regression models (or use multivariable regression) and find out which input variables have the biggest impact on each of dependent variable. I've found the function varImp from the caret package but I have no idea how to implement regression. There are target variables that reflect the weight of the stock-picking concept and the weights sum up to 1. Assuming I want to build a regression model with normalized Anuual Return as a dependent variable, how to implement these input variables that are ratios? Should I just omit one independent variable in order to estimate coefficients or transform them??

The second issue: Dependent variables have values from 0 to 1 (only six different values), what kind of regression should I implement? Which distribution is relevant?

I'm a beginner in R, so I would appreciate your help and simple hints :)


I'm not an R person, but I might be able to give you some conceptual guidance.

The ultimate question it seems you're asking is: What are the most important variables to predict Y.

Therefore, it sounds like you're worried about the different scales and types of measurements and how that might affect your ability to interpret the result.

In a way, you're correct. If you imagine a regression model, the coefficient for [Rel Win Rate] might be on a different scale than [Systematic Risk]. So, you wouldn't be able to compare the coefficients.

Instead, think about it this way: ignore the interpretation for now and just focus on building a model that predicts Y reliably. This might involve scaling variables for other reasons, but the only goal you should have is to build a reliable model.

Once you have a model, NOW you need to tackle the interpretation of what model is most important. Traditionally, different algorithms use different ways to assess Importance. A tree model might count the number of times a variable was split.

A model agnostic way to measure the importance of the variable is this:

  1. You already have a model
  2. Start with 1 variable, and destroy the information in it by totally shuffling just that column
  3. apply the model and measure how "accurate" it is
  4. compare that accuracy to the model's accuracy in step 1
  5. repeat for every column. The variables that mean the most will be observed by how much the model accuracy degrades during step 2&3

This concept is called Permutation Importance.

I hope this helps. In general, do what you need to do in order to get a decent model accuracy, and then worry about interpretation in a 2nd step.

  • $\begingroup$ Thanks Josh for your reply. I will analyze it and implement in this case. But supposing I'd like to build only one regression model for Annual Return, how should I implement these six independent ratio variables? $\endgroup$ Jun 5 '20 at 20:59
  • $\begingroup$ @AncymoniadesPereira I wouldn't do any special to them. Look up random forest regressor in R and try it as-is. $\endgroup$
    – Josh
    Jun 8 '20 at 17:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.