2
$\begingroup$

I am trying to fit a linear model to my data, which consist of a continuous dependent variable and several predictors, some continuous, some discrete and one dummy.

I used the R package glmnet, and the best R squared that I get is of 0.5.

Is there something that I could try to improve the model? Perhaps, I could scale the variables, but since I have a dummy variable, it seems that I should not.

$\endgroup$
1
  • 1
    $\begingroup$ It's hard to help with this little info. Maybe explain what your variables are, what you goals are, and post some data if possible. $\endgroup$
    – dimitriy
    Sep 27, 2013 at 20:28

1 Answer 1

7
$\begingroup$

Building a model shouldn't be driven solely on maximizing the R squared. You can include as many predictors as you can, square all the predictors and include them, cube them and add them, and so on... until you have as many predictors as you have samples and you can get a R square of 1.0. Obviously, you should never do this.

What you should do (most often) is include predictors you think are important, even before you look at the data. Make sure you plot the data. Take particular attention to the residual plot. It may tell you that you should transform your dependent variable, there is a curvilinear relationship, or that an outlier is reducing (or increasing) your R squared. You can then alter the model accordingly.

Scaling binary predictor variables (dummy variables are binary) will not explain additional variance. Meaning, the R sq will not change and the new variable is the same as the previous. Only thing different will be the slope.

$\endgroup$
6
  • 1
    $\begingroup$ Does it make sense to scale the dummy variables? $\endgroup$
    – dimitriy
    Sep 27, 2013 at 20:33
  • $\begingroup$ what do you mean by scale? $\endgroup$
    – Hotaka
    Sep 27, 2013 at 20:36
  • $\begingroup$ I interpret scaling as any variable transformation that changes the scale (normalizing, standardizing, logs, square roots or powers, etc.). The OP mentioned scaling the variables in the last sentence, but was worried that this did not make sense. Your statement that there's nothing special about dummy variables can be interpreted as advocating such transformations for them. $\endgroup$
    – dimitriy
    Sep 27, 2013 at 20:57
  • 1
    $\begingroup$ Oh, how did I miss that part? No, scaling binary predictor variables (dummy variables are binary) will not explain additional variance. Meaning, the R sq will not change and the new variable is the same as the previous. Only thing different will be the slope. $\endgroup$
    – Hotaka
    Sep 27, 2013 at 21:35
  • $\begingroup$ Unfortunately, I cannot reveal my data. Following the advice offered by Hotaka, I am trying to figure out whether there is a curvilinear relationship. Could you please direct me to some online tutorial or book (preferentially, using the R project)? $\endgroup$
    – PaulS
    Sep 27, 2013 at 21:37

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.