Skip to main content
14 votes

What is the difference between a "nested" and a "non-nested" model?

One model is nested in another if you can always obtain the first model by constraining some of the parameters of the second model. For example, the linear model $ y = a x + c $ is nested within the 2-...
Ruben van Bergen's user avatar
13 votes
Accepted

What is the relationship between ANOVA to compare means of several groups and ANOVA to compare nested models?

In my understanding, the abstract intuition of ANOVA is the following: One decomposes the sources of variance of the observed variable in various directions and investigates the respective ...
bmbb's user avatar
  • 308
10 votes

Comparing non nested models with AIC

It appears Akaike thought AIC was a useful tool for comparing non-nested models. "One important observation about AIC is that it is defined without specific reference to the true model [ f(x|kθ) ]. ...
JonesBC's user avatar
  • 111
9 votes

Prerequisites for AIC model comparison

Transformation of Variable In the analysis of time series it is common to try some kind of transformation on the variable. The decision on the choice of the transformation can be realized very simply ...
bjd's user avatar
  • 91
8 votes

AIC for nested models: is the AIC of a smaller model A always smaller than that of the nesting model B?

No, it is not true. The penalty term $2p$ may give you a worse AIC for the model with more parameters.
F. Tusell's user avatar
  • 8,608
8 votes

AIC for nested models: is the AIC of a smaller model A always smaller than that of the nesting model B?

To add on F. Tusell's answer: the AIC (but also other model selection criteria, such as the BIC or the HQC) contains two terms: A likelihood term, which measures the ability of the model to explain ...
Camille Gontier's user avatar
7 votes

Does likelihood ratio test control for overfitting?

Your reasoning is too pessimistic. Given the $K$ additional features, the LR test statistic will follow an asymptotic $\chi^2$ distribution with $K$ degrees of freedom if the null is true (and other ...
Christoph Hanck's user avatar
7 votes

What is the relationship between ANOVA to compare means of several groups and ANOVA to compare nested models?

If you are doing one-way ANOVA to test if there is a significant difference between groups, then implicitly you are comparing two nested models (so there is only one level of nesting, but it is still ...
Sextus Empiricus's user avatar
6 votes
Accepted

How to model nested data with repeated measures

A few things here: it probably doesn't make sense to treat test_type as a random effect. Test type may be a nuisance variable (i.e., you're not primarily ...
Ben Bolker's user avatar
  • 43.7k
5 votes
Accepted

Can one give an example(s) of when non-nested AIC model comparison is not useful for model selection?

Nesting is when all of the models tested can be derived by eliminating parameters from a parent model. Non-nesting is when the models contain parameters that are not in a set with subset(s) format. ...
Carl's user avatar
  • 13.1k
5 votes
Accepted

Exact equivalence of LR and Wald in linear regression under known error variance

Yes. [Remark: I took the liberty to ask and answer this question in order to be able to supply the missing parts in this answer. As I believe the present question could be of independent interest I ...
Christoph Hanck's user avatar
5 votes

Does likelihood ratio test control for overfitting?

How can we determine whether the added features cause the overfitting problem? It depends on what you intend to use your model for. If you indeed are interested in testing a hypothesis that the $K$ ...
Richard Hardy's user avatar
5 votes

Tests of Forecast Accuracy for Nested Models

The Diebold-Mariano test is well suited for the purpose it was designed for: to assess whether two forecasts produce loss of equal size in population. This holds regardless of the models that were ...
Richard Hardy's user avatar
5 votes

Multilevel between-subject design using glmer

It seems that you have indeed a crossed design. In particular, you would expect that answers from the same subjects are correlated, and that answers in the same questions will be correlated. In this ...
Dimitris Rizopoulos's user avatar
5 votes
Accepted

Lmer model syntax for a combination of crossed and nested random effects

Note here that you don't have crossed random effects. Here group is fully nested within district and ...
5 votes
Accepted

Mixed model with nesting

Changing your initial model from: m1 <- lmer(y ~ maturity * type + (1|run) + (1|run:block)) as correctly mentioned by @chl ...
Thomas Baumgartner's user avatar
5 votes
Accepted

how to compare the AUCs of ROC from nested models and non-nested models from published papers

The DeLong test requires the AUC-ROC covariance between the models A and B. If we are willing to accept that the covariance between the two models' AUC-ROC is 0, which is a down-right unreasonable ...
usεr11852's user avatar
  • 44.2k
5 votes
Accepted

Testing a Null Hypothesis Nested by the Alternative Hypothesis

By definition (see e.g. Jun Shao "Mathematical Statistics", 2nd edition, chapter 6, first paragraph) of statistical hypothesis tests, the null and the alternative hypothesis must be disjoint....
frank's user avatar
  • 10.8k
5 votes
Accepted

Does performing Likelihood Ratio Test to compare two nested LASSO models make statistical sense?

In the frequentist world of traditional statistics, penalized maximum likelihood is somewhat of an ad hoc procedure with limited inferential methods available after the fit. Likelihood ratio tests ...
Frank Harrell's user avatar
4 votes
Accepted

Are these glm models nested?

The models are not nested, as you correctly reasoned. For a model A to be nested in model B, you have to be able to get to A from B by removing parameters from B.
SlowLoris's user avatar
  • 1,086
4 votes
Accepted

Does the order of models (and so variables) matter in nested models?

Those are Type I or sequential sums of squares and yes, ordering generally matters. The sequence of tests is saying Given we have an intercept and added A what was the effect of doing that? Then, ...
Gavin Simpson's user avatar
4 votes
Accepted

Multilevel nested glmer model (logistic regression) with 4 groups

The default estimation procedure of glmer() is a Laplace approximation that is known not to be optimal for binary data. You could try using adaptive Gaussian ...
Dimitris Rizopoulos's user avatar
4 votes

Nested repeated measures linear mixed-effects model without time as a variable

Are the specifications in model1 correct to answer the question? No. The random structure (1|groups_2/record_id) does not make sense. Assuming that ...
Robert Long's user avatar
  • 60.9k
4 votes

How I can compare three models (nested) using lavaan with R?

SEM is (mean- and) covariance-structure analysis, so models are only comparable when fitted to the same data (i.e., the covariance matrix that the model reproduces must include all the same variables)....
Terrence's user avatar
  • 2,138
3 votes
Accepted

Testing nested models with clustered errors

So is it completely wrong to use the Likelihood Ratio in this case? Saying "completely wrong" is too strong, because it might still have in some cases better small sample properties than a cluster ...
Josef's user avatar
  • 3,212
3 votes

Does the order of models (and so variables) matter in nested models?

Gavin gave the correct answer about what is going on here and why the order is relevant (+1). I will briefly focus on the side question: "what can I do to ensure that my results are not biased or ...
usεr11852's user avatar
  • 44.2k
3 votes

Does likelihood ratio test control for overfitting?

I think you are looking for one of the information criteria such as AIC or BIC which penalise you for adding parameters. https://en.wikipedia.org/wiki/Akaike_information_criterion has some discussion ...
mdewey's user avatar
  • 17.8k
3 votes
Accepted

How to compare ordered logistic nested models?

Use the likelihood ratio test. Twice the difference in log likelihood (larger minus smaller) will be approximately $\chi^2_1$ distributed in this case. Generally, the degrees on freedom equals the ...
not_bonferroni's user avatar
3 votes

Are any pairs of these four structural models nested in another?

a) is the saturated model. All models are nested within the saturated model. That's the basis of the chi-square test. You're testing against the saturated model, which has 0 df and 0 chi-square. c) ...
Jeremy Miles's user avatar
  • 17.9k
3 votes

Comparing non-nested GLMMs with AIC

Yes, you can compare these models with AIC. You may have to fit m2 and m3 with maximum likelihood to do so. (The default method for lmer is REML.) Be sure to inspect graphs of model fits.
JTH's user avatar
  • 1,033

Only top scored, non community-wiki answers of a minimum length are eligible