7
$\begingroup$

I have dataset with 9524 observations / 97 variables.

Most of variables are numerical, and some of factor variables (Yes/no or several levels)

I want to perform multiple linear regression with this dataset.

  1. First step was to do log transformation on the data, since the data is highly skewed. (I did log(x+1) because of many have 0)

These are the histograms of my data after log transformation. enter image description here

"API05B" will be dependent variable for the linear regression.

There's more variables, and most of them are heavily skewed. (mostly right or some left)

  1. Anyway, I tried to keep performing regression to see the results. With 'regsubsets' - forward selection, I tried to select best predictors (or best model) among those variables.

enter image description here enter image description here

I chose the number of predictors when showing significant changes of BIC or Cp (size of bias), which the number of predictors was 3.

The following is the histogram of the variables selected by regsubsets.

enter image description here

I tried to not take care of the distribution of variables, since there's no assumptions for the distribution of data in linear regression.

But, I worried about the skewness would impact on the heteroskedasticity.

  1. Diagnosis for the linear model - outliers / multicollinearity / heteroskedasticity

I removed outliers / correct the multicollinearity with VIF.

There was no problem with these, but..

I did lmtest::bptest / lmtest::coeftest, the result saying the heteroskedasticity exist.

enter image description here enter image description here

Box-cox transformation didn't work as well.

Here's the summary plot of my final model.

enter image description here

I've read several articles about dealing with skewed data or heteroskedasticity,

most of them saying log transformation or box-cox transformation would be helpful, but it didn't work..

Some of them recommend to not stick to linear regression, such as trying robust linear regression / zero-inflated model / two-part models etc..

Issues I want to solve..

  1. dealing with skewness of data or heteroskedasticity

    • another transformation needed?

  2. predictors selection

    • regsubsets or lasso ?
    • transformation first? or selecting predictors first?

  3. another approaches needed?

    • If none of the above would be helpful for this issue, another approaches needed as mentioned above?

  4. Is there anything wrong in my process?

Any tips or suggestions will be appreciated!! Thank you!

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Something to keep in mind, the heteroskedasticity test need to be taken with a grain of salt, especially for large samples, since the test will in these cases often times easily reject the null. The distributions of your variables does not matter, the residual plots look mostly fine, except for the decreasing variance towards the right end. Have you tried building a model without transformations? $\endgroup$
    – user2974951
    Oct 26, 2018 at 5:14
  • $\begingroup$ Yes,I've built the model without transformation. I guess everything worked well, but only the heteroskedasticity problem. $\endgroup$
    – David Kwon
    Oct 28, 2018 at 22:22
  • $\begingroup$ Depending on what linear model you used, you could: 1) use residualPlots() from the car package to see which variables are causing problems, 2) use weights argument from the nlme package to model the variance. $\endgroup$
    – user2974951
    Oct 29, 2018 at 8:44

1 Answer 1

Reset to default
2
$\begingroup$

There are too many questions asked. You are welcome to break it down. And many of the questions are already answered well in this forum.

I will only address your first question here.

There's more variables, and most of them are heavily skewed. (mostly right or some left)

It is seems you may have some mis-understandings on linear regression assumptions. Linear regression does not assume independent variable / model input to be Gaussian distributed, but assume the residual.

Details can be found

Why is the normality of residuals "barely important at all" for the purpose of estimating the regression line?

Why linear regression has assumption on residual but generalized linear model has assumptions on response?

In the first link I provided, it also explains normality of residuals is not that important as you may think.

For feature selections see here

Improve this answer
$\endgroup$
1
  • $\begingroup$ If two other answers basically say the same thing as your answer, might it not be better to vote to close as duplicate? Alternatively if you didn't do that because OP asks too many questions, shouldn't you vote to close as too broad? $\endgroup$
    – Glen_b
    Oct 26, 2018 at 12:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.