I've run 4 models (simple LM, quadratic model, GLMM, and GLMM with quadratic) to predict tree age (age) from tree diameter (D) for each of 42 species (SPEC). The diameter data has all been log transformed to account for non-normality and heavy weighting of near-zero values. I've compared each of these models using AIC separately for each of the species and chose the model with the lowest AIC to use for each species.
I have a rough idea about what tree age predictions should look like (i.e., not super large or negative). The problem is that most of my 'best' models (generally the GLMM with quadratic) make essentially nonsense predictions for that species. However, if I look at the predictions of 'worse' models (typically the simple LM) for those given species, the predicted age values make more sense. So I would be more inclined to use these 'worse' models in these instances. The 'worse' model that is the best predictor is further not always consistent between species and some species' ages are best predicted from the model with the lowest AIC -- in other words, it's not consistent.
My question is: Is there a methodological way to choose the model post hoc regardless of how it ranks in terms of AIC (and instead by how it predicts sensible values with new data)?