# Visually comparing two models based on the per-observation AIC

In order to visually compare two models (logistic regression, in case that it matters) I thought of plotting the contribution of the individual observations to the AIC of the respective model. The plot looks like this:

One can see that, although for some observations the full model makes larger errors than the simple one (i.e. a larger contribution to the AIC), for the majority of the observations the per-observation AIC is lower for the full model. The linear regression (the orange line) reflects that by being below the diagonal dotted line. This, I hope, gives some visual support for using the full model---assuming that the model selection is based on the AIC in the first place.

Are plots like these actually used in practice and do they have an established name?

If not, why? Is this plot in any way misleading or unclear?

• Linear regression is not symmetrical. If you regressed X on Y, would the orange line be above the diagonal? Moreover, what is the justification for treating X as fixed & Y as (partly) random? Mar 20, 2023 at 16:27
• If the idea is to argue for the full model, what is the advantage of this method over just reporting that the full model's AIC is lower? Is someone arguing that the lower AIC is due to just some of the data or something? Mar 20, 2023 at 16:29
• @gung-ReinstateMonica (1): Yes, since the regression line must pass through the $(\bar{x}, \bar{y})$. Regarding randomness, if it were for predictive modelling, I'd worry more about the non-normality and heteroskedacticity of the errors. But I'm just interested in an intuitive visualisation. Mar 21, 2023 at 7:55
• @gung-ReinstateMonica (2): This is just a general example. In my case at hand I have the opposite situation: The full model has a slightly lower AIC than a simpler one (467 vs. 469, less than 0.5% difference). I want to argue that the simpler model is sufficient; that the difference in AIC is likely random. The graph shows the observations neatly aligned along the diagonal. Mar 21, 2023 at 7:57
• Good point regarding $(\bar x, \bar y)$, but I bet it would cross the diagonal. Is the simpler model nested within the full model (e.g., the full model has X1-5, & the simpler model only has X1)? Mar 21, 2023 at 11:33

anova(simple.model, full.model, test="LRT")