Choosing transformation for variable: Should AIC factor into this decision?

Question

I have built a series of Generalized Linear Mixed Models, in order for the models to converge I need to transform my continuous explanatory variables. Both log transformation and scaling/centering work. They produce models with different AICs, and different log odds.

Using the exact same variables and data, with a log transformation it has a lower AIC than the scale transformation. Looking at the residuals and other model diagnostics- the models are both fine. Should I be choosing the model with a lower AIC, based on transformation?

Answer 1

It depends on your goals. If you're just trying to understand the relationships in your data, it absolutely makes sense to use whatever transformations best makes sense of the data. As a side note, since the number of parameters doesn't change, changes in AIC just reflect changes in model log-likelihood.

However, if you're building a predictive model or doing hypothesis testing this may be a problem.

For predictive modelling, since the transformation you use depends on the training data this can cause overfitting, as the same transformation may not be appropriate for the data you make predictions on.

For hypothesis testing, similarly, you've chosen the transformation that leads to the best linear relationship between the predictor and the outcome, and doing so increases the probability of a type 1 error. In other words, if a predictor is actually unrelated to the outcome, you'll normally only find $p < .05$ 5% of the time, but it will occur more often if you always choose the best-fitting transformation.