What is the actual significance of a difference in AIC or BIC values? Usually, when a difference of a statistic is discussed, that discussion is presented in the context of a significance of that difference. When self-entropy, i.e., information content, is examined, especially, but not only when non-nested models are compared, we use the lower value of the AIC, AICc, BIC or other information content index to suggest what the better model is. However, more generally, entropy is case-wise, i.e., data-wise, variable.
Question With what certainty do we know, based on comparative information content indices from a particular data set, that that lower index value properly suggests the correct model more generally for a less limited data set?
I feel that non-nested model comparison of information content is not always relevant in all circumstances, for example see this Q/A. Nesting is when all of the models tested can be derived by eliminating parameters from a parent model. Non-nesting is when the models contain parameters that are not in a set with subset(s) format.
I really would appreciate any insight into the variability of comparison of information content for either nested or non-nested models in the context of subset data.
 A: The difference in AIC (or BIC) for two models is twice the log-likelihood ratio minus a constant: it follows immediately that in any particular case selecting the AIC corresponds to performing a likelihood-ratio test, but that in different cases it corresponds to tests of different significance levels.
With nested models, the null hypothesis has to be that the smaller model holds. Given some regularity conditions, Wilks' theorem applies; so if $p$ is the difference in the number of free parameters between the models, asymptotically the probability of AIC's selecting the larger model when the smaller one in fact holds is the probability that a chi-squared r.v. with $p$ degrees of freedom exceeds $2p$. For $p=1$ the significance of the test is 0.157; for $p=2$, 0.135; & so on. When exact tests are possible the distribution of the log-likelihood ratio of course depends on precisely what the models are.
With non-nested models, even finding an asymptotic distribution for the log-likelihood ratio involves the calculation of rather complicated expectations under the null (see Cox's or Vuong's papers referenced in Generalized log likelihood ratio test for non-nested models & Comparison of log-likelihood of two non-nested models). I doubt much can be said in general about the significance of a difference in AIC.
The moral has already been given, pithily, by @RichardHardy:

How do you define what is a "correct model"? Is it the data generating process (DGP)? If so, why would you be using AIC trying to identify the DGP? The question AIC is answering is not "Which of the models is the DGP?". Try asking a different question, such as "Which model will give better predictions under a certain type of loss (associated with the likelihood being used)?", and you might find that AIC is answering "correctly" (or perhaps not?). That is, use a hammer for hammering nails

Problems with the AIC include its accuracy in small samples (the bias-correction term is only to first order) & when neither model is particularly close (in the sense of Kullback–Leibler divergence) to the true model; but it can't fairly be criticized for not doing what it wasn't made for: there are hypothesis tests for that.
