0
$\begingroup$

I am a student working on a project whose main subject is variable selection through methods such as AIC stepwise regression, Ridge regression and LASSO regression using R.

My aim is to build a simple and efficient model which tries to predict the response given a data set which contains values for 44 predictor variables.

Before starting my modelling, I wanted to know if transformations are needed and if outliers need to be removed. As there are many covariates, I don't have time to go through each case exhaustively, so, being my first time working on this kind of projects, I don't know what to do.

Is there any simple and justifiable way to assess the need for a transformation or outlier removal in these cases? Are there any commonly used rules of thumb?

Any other suggestion or comment is highly appreciated, thank you very much.

$\endgroup$
0
$\begingroup$

My first thought is that 44 variables is not so many that you cannot go through each one in some detail.

Second, you will probably also want to look at which variables can be deleted due to colinearity.

Finally, to your question, I would advise against any automatic variable transformations or outlier removal. For one thing, OLS regression makes no assumptions about the shape of the distribution of the variables, only about the errors. For another, outliers are often the most interesting points (unless they are data entry errors) and, for a third, if you do violate the assumptions of OLS regression, I think it is better to use a different method (e.g. robust regression, quantile regression) that makes fewer assumptions rather than transform your variables.

$\endgroup$
  • $\begingroup$ Thank you very much for your answer. I will take in consideration your suggestions. However, I have some questions. How can I justify deleting variables due to collinearity? Is it enough to delete variables that are highly correlated or should I fit a model and evaluate the VIFs? Concerning the transformations, I am not familiar with robust and quantile regressions, and I am explicitly asked to use of the methods I listed above. Thus, if I notice skewness should I apply transformations? $\endgroup$ – A-B-izi Feb 17 '18 at 14:48
  • $\begingroup$ You can justify deleting vairables that are highly colinear (which is not exactly the same as highly correlated) by noting that they add little information. If you have to use OLS regression, then you might have to do some transformation but it should be based on the residuals, not the independent variables. $\endgroup$ – Peter Flom - Reinstate Monica Feb 17 '18 at 15:11
  • $\begingroup$ If i was automating this, I might have one thing be "OLS is not appropriate here, please find an expert". $\endgroup$ – Peter Flom - Reinstate Monica Feb 17 '18 at 17:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.