0
$\begingroup$

I have a lognormally distributed continuous dependent variable that I would like to predict using multiple regression. I am using a forward selection process and have selected three predictor variables at the moment:

  • x1: A continuous variable with approximately normal distribution and slight negative skew
  • x2: A continuous variable with normal distribution
  • x3: A categorical variable with 14 categories.

If it helps, here are some plots:

x1 vs. y x2 vs. y x3 vs. y

I have two questions:

  1. Should I log-transform the dependent variable to make it normally distributed before feeding into a regression?
  2. With or without the transformation, what are some basic rules of thumb for determining what kind of (multiple) regression to use?

Some extra points:

  • There is no pairwise collinearity between the predictor variables.
Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

First, you shouldn't use forward (or backward or stepwise) selection. They all have major problems, this has been discussed here many times. You seem to have a very large sample, this means that even weak relationships will be significant.

Second, in my opinion, whether you should transform Y by taking logs should depend on the substantive question, rather than the data itself. What is Y? If it is some sort of money variable (income, expenses, wealth, cost of an item) then taking logs often makes sense.

Third, there do seem to be nonlinear relationships; there might also be heteroskedasticity and maybe some outliers. Whether the residuals are close to normal can't be assessed on what you've shown, but maybe it's there too. For the nonlinearity, you can look at spline effects of the IVs, in R there is the rcs function in the rms package. In SAS there is PROC TRANSREG (ADAPTIVEREG also does splines, but TRANSREG seems more useful here, with so few variables). For the heteroskedasticity and possible nonnormality of residuals, you can use quantile regression (see the quantreg package in R or PROC QUANTREG in SAS.

Improve this answer
$\endgroup$
3
  • 1
    $\begingroup$ Thank you, I'll do away with feature selection on your advice. $Y$ represents the ultimate tensile strength (UTS) of a material, and the underlying question is whether $Y$ can be predicted from available data. Various machines with potential measurement errors calculate each $x_i$, but domain knowledge suggests that $x_1$ is the most useful. Since there may be measurement errors, the prediction itself might be a distribution (is this a density estimation problem?) for a chosen range of $x_1$, say. It does seem to be non-linear and heteroscedastic - thanks for the resources and expertise. $\endgroup$
    – user201118
    Oct 17, 2019 at 12:46
  • $\begingroup$ Peter, do you think that taking a log of $Y$, the ultimate tensile strength, makes sense? And from my comment above, could this turn into a density estimation problem if the heteroscedasticity cannot be resolved? $\endgroup$
    – user201118
    Oct 18, 2019 at 2:16
  • $\begingroup$ I don't know if log(tensile stregnth) makes sense - that's a question for you (or a metallurgist) not a statistician. Do you think about tensile strength additively or multiplicatively? $\endgroup$
    – Peter Flom
    Oct 18, 2019 at 12:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.