# Equivalence of AIC and LOOCV under mismatched loss functions

Under certain conditions, AIC and LOOCV (leave-one-out cross validation) are asymptotically equivalent (Stone, 1977). Stone's paper is less than 4 pages long, but quite mathy, so I turn here for some assistance. I presume the equivalence holds when the loss function in LOOCV is exactly the same or somehow compatible with the loss function implied by the likelihood as used in the AIC.

Questions

1. What happens if the loss function employed in LOOCV does not exactly correspond to the loss function implied by the likelihood in AIC?
For example, say that the loss function in LOOCV is some form of tick function (quantile loss) while the likelihood is normal?
2. Under what conditions, at least roughly, would the asymptotic equivalence hold?
Simple, concrete examples as well as rigorous explications are welcome.

References

• – Richard Hardy May 8 '19 at 14:40
• Related question: Example and counterexample for Stone's (1977) assumption. – Richard Hardy May 8 '19 at 15:00
• Somewhat related question: Optimality of AIC w.r.t. loss functions used for evaluation. – Richard Hardy Sep 10 '19 at 19:08
• Hm. Aren't we comparing apples to oranges here? Information criteria assess entire likelihoods, and quantile losses assess point forecasts. So the idea would be to obtain a predictive density, then extract the quantile from that, and to assess this using the quantile loss. I see no conflict. An (IMO) more interesting question would be to compare AIC to cross-validation using density predictions assessed through proper scoring rules. What do you think? – Stephan Kolassa May 10 at 8:31
• @StephanKolassa, thank you for a thoughtful discussion. I can easily picture that users are confused about their loss functions! Regarding FIC, the corresponding Wikipedia article contains the main references. Besides that, you may follow the work of Gerda Claeskens; she and her students have done quite a bit more on FIC since 2003. – Richard Hardy May 10 at 9:38