1
$\begingroup$

I have a dataset with $5000$ observations, and 10 explanatory and 1 response variable (binary 0 or 1), and my task is to make a logistic regression model for prediction (but also needs to provide some understanding of the situation), and right now I am at the EDA stage.

My issue is as follows, all explanatory variables are extremely skewed, some only have positive values which lend themselves nicely to a log transform, while others include negative values, zero values or both. There are several methods and transformations I came across, as well as the bestNormalize package in R, and the top transformations I found were Yeo-Johnson, arcsinh, power transforms (such as cube root) and orderNorm. One final suggestion from the package is that for variables with zero and positive values, one should add a small factor and then log it.

orderNorm performs the best, however it is difficult to interpret and apparently it has issues when using it on new unseen data. Power transforms were very weak, and Yeo-Johnson (extension of Box-Cox) seems like the best candidate so far. Arcsinh doesn't seem to have any glaring weaknesses either.

Given this, I wanted to ask for advice and possible suggestions. In particular about the case where you add a small factor, and then log transform. Many sources state this introduces bias and is in general bad practice, while others argue the effects are negligible. I have added a density plot below to show just how severe the problem is.

One other questions I have, is about missing values. I am aware of the classifications MCAR, MAR and MNAR, and how it is virtually impossible to distinguish between MAR and MNAR. I wanted to ask if there is ever a reason to delete observations (for MCAR), as opposed to using imputation.

enter image description here

$\endgroup$
3
  • 1
    $\begingroup$ Why are you transforming them at all? Logistic regression does not require normal data. $\endgroup$
    – Peter Flom
    Commented Mar 23 at 21:25
  • $\begingroup$ Wouldn't extreme skewness introduce problems when I am constructing the model? I only worked with linear regression so far where this was an issue. $\endgroup$ Commented Mar 23 at 21:31
  • $\begingroup$ See Shawn's answer. $\endgroup$
    – Peter Flom
    Commented Mar 24 at 9:29

1 Answer 1

1
$\begingroup$

I originally voted to close this question because it contains a lot of different comments and questions, but coming back to this there are only two real questions here which are directly answerable, the problem of transformation and the problem of deleting missing values.

First, there is absolutely no reason to care about this:

My issue is as follows, all explanatory variables are extremely skewed, some only have positive values which lend themselves nicely to a log transform, while others include negative values, zero values or both.

Because the regression modeling depends on the distribution of the residuals, which are already modeled explicitly with the logit link function in logistic regression, there is no reason to transform in this case. With respect to deleting missing values:

I wanted to ask if there is ever a reason to delete observations (for MCAR), as opposed to using imputation.

Generally speaking it is a bad idea, but if you can clearly identify that it was an erroneous entry, then it is okay to fix or delete the value. For example, we may accidentally create missingness by pivoting data which doesn't have unique values, or because a statistical software attributes an incorrect entry as missing based off some coding (e.g. NA values = 999, but somebody accidentally enters 999 when the value was supposed to be 99).

In all other cases, there is plenty of literature that says it is an otherwise bad practice to simply employ listwise deletion.

$\endgroup$
2
  • 1
    $\begingroup$ I see, thank you very much! I was under the impression that extreme skewness is a big issue for logistic regression, like with OLS, but it appears this is not the case. $\endgroup$ Commented Mar 23 at 21:37
  • $\begingroup$ Even in standard OLS regression this can be a non-issue (particularly for predictors), but that depends a lot on the data entered into the model. This matters a lot less for logistic regression. $\endgroup$ Commented Mar 23 at 21:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.