Timeline for Logic behind the ANOVA F-test in simple linear regression
Current License: CC BY-SA 2.5
13 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Aug 17, 2022 at 23:32 | comment | added | Glen_b | @JohnJiang You shouldn't necessarily expect that the pieces correspond in that exact sense. While it may well be possible to derive some common test from a likelihood ratio test, and while both things may involve ratios, the numerators and denominators of both will not typically be the same things. Rather, in the circumstances where it works exactly. their test statistics will be (strictly) monotonically related, so you will have two equivalent tests (they must reject or not reject the same samples at the same significance level). | |
| Sep 24, 2015 at 23:50 | comment | added | John Jiang | @chl: Thanks for the answer. I am wondering how exactly the F-test is a likelihood ratio test, that is, why the enumerator is the likelihood of the null model and the denominator the likelihood of the alternative model. | |
| Mar 16, 2011 at 0:10 | comment | added | probabilityislogic | @chl - I think the problem is with leading with "intuition" and then seeing if it "makes sense" - it gets you into trouble when you try to "generalise". this is because one is "generalising" the intuition rather than generalising the mathematics. The SS in a additive model with normal errors have a logical justification - which is why they work so well. However, if your interested in prediction or model selection - I think the logic "adds in" some extra pieces of information (such as an "occam" effect). | |
| Mar 15, 2011 at 20:34 | comment | added | chl | @probabilityislogic (Con't) or a simple regression has a very intuitive approach in term of projection, subspaces, etc. I'll try to update my response to reflect this. Feel free to comment on. | |
| Mar 15, 2011 at 20:33 | comment | added | chl | @probabilityislogic Good point. My idea was originally to show the logic behind model comparison, of which the simple regression model is just a particular case (compare to the "very null" model), which also motivates the quick note about LRT. I agree with you, if we work along the line of a pure Neyman-Pearson approach for HT. However, I was mainly thinking in terms of the Theory of LMs, where SS have a direct geometrical interpretation and where model comparison or the single F-test for a one-way ANOVA (...) | |
| Mar 15, 2011 at 13:23 | comment | added | probabilityislogic | @chl - A bit of nitpicking from me. It is a nice answer about the intuition behind the F-test, and how it "goes in the right directions". But it doesn't explain the logic of why you should choose this particular test. For example, why shouldn't we use the PRESS statistic? You hinted at the likelihood ratio - which does have a logical justification - hence its applicability to all models, unlike the F-test. | |
| Mar 14, 2011 at 11:35 | comment | added | chl | @Chase I just rediscovered this very nice response from @Gavin about the Interpretation of R's lm() output. | |
| Mar 14, 2011 at 9:11 | comment | added | chl |
@Chase Yes, the ANOVA Table I have in mind are arranged in this way too. Feel free to ask the question; I'd love to hear what other users think of that. I generally use anova() for GLM comparison. When applied to an lm or aov object, it displays separate effects (SS) for each term in the model and doesn't show TSS. (I used to apply this the other way around, namely after fitting an ANOVA with aov(), I can use summary.lm() to get an idea of treatment contrasts.) However, there're subtle issues between summary.lm() and summary.aov(), especially related to sequential fitting.
|
|
| Mar 14, 2011 at 0:54 | comment | added | Chase |
@chl - First of all, nice answer! This may warrant it's own question so let me know...but the descriptions I have read about ANOVA tables for regression models typically refer to three rows in the table: predictors, errors, and total. However, the anova() function in R returns an individual row for each predictor in the model. For instance, anova(lm0) above returns a row for V1, V2, and Residuals (and no total). As such, we get two F* statistics for this model. How does this change the interpretation of the F* statistic reported in the ANOVA table?
|
|
| Mar 13, 2011 at 21:38 | history | edited | chl | CC BY-SA 2.5 |
added 136 characters in body
|
| Mar 13, 2011 at 21:32 | history | edited | chl | CC BY-SA 2.5 |
added 165 characters in body
|
| Mar 13, 2011 at 21:10 | history | edited | chl | CC BY-SA 2.5 |
added 204 characters in body
|
| Mar 13, 2011 at 21:04 | history | answered | chl | CC BY-SA 2.5 |