In linear regression, the transformations of explanatory variables is done to have maximum correlation with the dependent variable.
What is the best measure of choosing between multiple transformations in logistic regression as dependent variable is binary and not continuous?
The end goal is to maximize the lift (predictive power) of the model.
The optimality criterion used by logistic regression (and many other methods) is the likelihood function. It is used to estimate $\beta$ including multiple $\beta$ representing one $X$ to achieve quadratic, cubic, and piecewise polynomial (spline) fits. It can also be used to choose from among competing transformations of $X$ but the act of choosing will not be reflected in the information matrix, so the resulting variance of $X\hat{\beta}$ will be too small, making confidence intervals not have the stated coverage probability. If you make transformation estimation an explicit goal of model fitting (and regression splines are excellent ways to do this) you will preserve all aspects of statistical inference. Depending on the sample size, a restricted (linear in both tails) cubic spline with 4 knots, requiring 3 parameters, can be a good choice.
rms lrm function for quadratic penalization). As you said, it is best to estimate transformations in an adjusted rather than a univariate fashion.
$\endgroup$
Commented
Jul 15, 2013 at 11:46
No, in linear models the transformation is not (or ought not) be done to have maximum correlation with the dependent variable. It should be done to either a) Meet model assumptions about the residuals or b) Have a more sensible explanatory variable; that is, one that makes sense, substantively. As @Andy points out, this may not be sufficient. But, in that case, I'd then look for an alternate method of regression (see below) rather than take some weird transformation. E.g. a model such as $Y = b_0 + b_1x_1^{.21} + b_2x_2^{.73}$ is going to be a mess to explain.
In logistic regression (at least, in dichotomous logistic) there are fewer assumptions (and none about the residuals, as far as I know), so only b) applies.
Even for linear models, I'd favor using b). And then, if the assumptions aren't met, using some other form of regression (could be robust regression, could be a spline model, could be polynomials).
have a more sensible explanatory variable is quite ambiguous and should be expanded upon. I would typically take it to mean a transformation that allows easier interpretation of the regression coefficients, but that is obviously not in and of itself sufficient (for either OLS or Logistic regression).
$\endgroup$
With generalized linear modeling the mathematical measure that is minimized is called the "deviance" (-2*log-likelihood). There are several sorts of residuals that can be developed. The "deviance residuals" are the individual terms in a modestly complex expression. I think these a most understandable when applied to categorical variables. For a categorical variable using logistic regression these are just the differences between the log-odds(model) and log-odds(data), but for continuous variables they are somewhat more complex. Deviance residuals are what are minimized in the iterative process. See this description at the UCLA website for some nice plots of deviance residuals.
It looks to me that analysis of "lift" is done on the scale of probabilities, rather than on the log-odds or odds scale or likelihoods. I see that Frank Harrell has offered some advice and any perceived dispute between Frank and I should be resolved by massive weighting of Frank's opinion. (My advice would be to buy Frank's RMS book.) I'm surprised he didn't offer advice to consider penalized methods and that he didn't issue a caution against over-fitting. I would think that choosing a transformation simply because it maximized "lift" would be akin to choosing models that maximized "accuracy". I know he doesn't endorse that strategy.
