I have dataset with 9524 observations / 97 variables.
Most of variables are numerical, and some of factor variables (Yes/no or several levels)
I want to perform multiple linear regression with this dataset.
- First step was to do log transformation on the data, since the data is highly skewed. (I did log(x+1) because of many have 0)
"API05B" will be dependent variable for the linear regression.
There's more variables, and most of them are heavily skewed. (mostly right or some left)
- Anyway, I tried to keep performing regression to see the results. With 'regsubsets' - forward selection, I tried to select best predictors (or best model) among those variables.
I chose the number of predictors when showing significant changes of BIC or Cp (size of bias), which the number of predictors was 3.
The following is the histogram of the variables selected by regsubsets.
I tried to not take care of the distribution of variables, since there's no assumptions for the distribution of data in linear regression.
But, I worried about the skewness would impact on the heteroskedasticity.
- Diagnosis for the linear model - outliers / multicollinearity / heteroskedasticity
I removed outliers / correct the multicollinearity with VIF.
There was no problem with these, but..
I did lmtest::bptest / lmtest::coeftest, the result saying the heteroskedasticity exist.
Box-cox transformation didn't work as well.
Here's the summary plot of my final model.
I've read several articles about dealing with skewed data or heteroskedasticity,
most of them saying log transformation or box-cox transformation would be helpful, but it didn't work..
Some of them recommend to not stick to linear regression, such as trying robust linear regression / zero-inflated model / two-part models etc..
Issues I want to solve..
dealing with skewness of data or heteroskedasticity
- another transformation needed?
- regsubsets or lasso ?
- transformation first? or selecting predictors first?
another approaches needed?
- If none of the above would be helpful for this issue, another approaches needed as mentioned above?
Is there anything wrong in my process?
Any tips or suggestions will be appreciated!! Thank you!