11
$\begingroup$

I have large survey data, a binary outcome variable and many explanatory variables including binary and continuous. I am building model sets (experimenting with both GLM and mixed GLM) and using information theoretic approaches to select the top model. I carefully examined the explanatories (both continuous and categorical) for correlations and I am only using those in the same model that have a Pearson or Phicorr coeff less than 0.3. I would like to give all of my continuous variables a fair chance in competing for the top model. In my experience, transforming those that need it based on skew improves the model they participate in (lower AIC).

My first question is: is this improvement because transformation improves the linearity with the logit? Or is correcting skew improves the balance of the explanatory variables somehow by making the data more symmetric? I wish I understood the mathematical reasons behind this but for now, if someone could explain this in easy terms, that would be great. If you have any references I could use, I would really appreciate it.

Many internet sites say that because normality is not an assumption in binary logistic regression, do not transform the variables. But I feel that by not transforming my variables I leave some at disadvantage compared to others and it might affect what the top model is and changes the inference (well, it usually does not, but in some datasets it does). Some of my variables perform better when log transformed, some when squared (different direction of skew) and some untransformed.

Would someone be able to give me a guideline what to be careful about when transforming explanatory variables for logistic regression and if not to do it, why not?

Improve this question
$\endgroup$
2
  • 2
    $\begingroup$ Indeed, in logistic regression there is no assumption of normallity (or logistic distribution in this case). The link function $F$ (sometimes denoted $F^{-1}$) is used for modelling the relationship between the probability of observing a $1$ ($0$) with the covariates through $P(Y=1\vert \beta, X)=F(X\beta)$. A poor fit/performance may be due to the choice of the link function. An alternative for sorting this issue consists of using a more flexible distribution, see for example this paper. $\endgroup$
    – user10525
    Commented Oct 4, 2012 at 16:45
  • $\begingroup$ Although written in a different context, much of what you are asking for is in my answer (or in the links in my answer) here: Are normally distributed X and Y more likely to result in normally distributed residuals? $\endgroup$
    – gung - Reinstate Monica
    Commented Aug 28, 2014 at 19:40

2 Answers 2

Reset to default
3
$\begingroup$

You should be wary of decide about transforming or not the variables just on statistical grounds. You must look on interpretation. ¿Is it reasonable that your responses is linear in $x$? or is it more probably linear in $\log(x)$? And to discuss that, we need to know your varaibles... Just as an example: independent of model fit, I wouldn't believe mortality to be a linear function of age!

Since you say you have "large data", you could look into splines, to let the data speak about transformations ... for instance, package mgcv in R. But even using such technology (or other methodsto search for transformations automatically), the ultimate test is to ask yourselves what makes scientific sense. ¿What do other people in your field do with similar data?

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for supporting my worries: indeed, I have though about what makes biological sense. The problem is, that I actually have two related datasets and I would like to draw conclusions from both at the same time. But in one subset, the density variable is best in the models untransformed while in the other log transformation is the best. Log transformation improves the relationship in the dataset that has the lower values for that variable, so it will be very difficult to reconcile these two datasets I think, unless I leave the variable untransformed in both. $\endgroup$
    – Zsuzsa
    Commented Oct 10, 2012 at 18:51
  • 1
    $\begingroup$ The experts in a field are seldom capable of knowing apriori the "right" transformations for variables. I almost never see linear relationships so when the sample size warrants I relax this assumption using regression splines. I make the result interpretable with pictures. $\endgroup$
    – Frank Harrell
    Commented Aug 28, 2014 at 18:49
3
$\begingroup$

The critical issue is what are the numbers supposed to represent in the real world and what is the hypothesized relationship between those variables and the dependent variable. You may improve your model by 'cleaning' your data, but if it doesn't better reflect the real world you have been unsuccessful. Maybe the distributions of your data mean your modeling approach is incorrect and you need a different approach altogether, maybe your data have problems.

Why you remove variables if they have corr>.3 is beyond me. Maybe those things really are related and both are important to the dependent variable. You can deal with this with an index or a function representing the joint contribution of correlated variables. It appears you are blindly throwing out information based on an arbitrary statistical criteria. Why not use corr>.31, or .33?

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.