Timeline for Moderate test statistic but low p-value in an F-test
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Oct 8 at 6:42 | vote | accept | Richard Hardy | ||
| Sep 26 at 14:09 | comment | added | Richard Hardy | @LukasLohse, thank you for helping me! I am satisfied with the explanations but will wait a little bit before accepting your answer. | |
| Sep 26 at 13:48 | comment | added | Lukas Lohse | @RichardHardy the sum of squares is a scaled chi-square, based on the restricted RSS of 38 (most of the variance was already explained by X2). By pure chance we expect this to split 18/80, so the OLS F-statistic is (6.8739/31.982)/(18/80) = 0.9552, which then get's inflated to 6.8 | |
| Sep 26 at 13:02 | comment | added | Christoph Hanck | At least (unlike in the present DGP) when there actually is heteroskedasticity, conventional wisdom has it that the robust s.e.s typically (although not necessarily, see e.g. stats.stackexchange.com/questions/627057/…) are larger to correct upward size distortions of conventional s.e. estimators. E.g., sciencedirect.com/science/article/abs/pii/0304407685901587 should have discussion. | |
| Sep 26 at 6:59 | comment | added | Richard Hardy | I am asking about my particular example just to have concrete numbers to work with. Since the sum of squares is about equal the F-stat in my case, and since there are 18 degrees of freedom, does that not mean an 18-fold ratio of vanilla vs. NeweyWest variances? | |
| Sep 26 at 6:55 | comment | added | Lukas Lohse | @RichardHardy where are you getting 18 from? Individual standard errors are about 0.85 of the OLS estimate and this, somehow and also including the covariances, combines over 20 estimators to an 5-fold increase in F-statistic, although this has a lot of spread on the log-scale. To see that 6.8 was a coincidence rerun your code with seed 9998. | |
| Sep 26 at 5:47 | comment | added | Richard Hardy | Let me see if I got you right. The only problem here is that Newey-West severely underestimates the standard errors (by about $\sqrt{18}$ fold in my example)? And that the sum of squares (under vanilla variance estimator) approximately equaling the $F$ statistic under Newey-West is a coincidence (and an example of how severe the underestimation can be, here $18$ fold)? | |
| Sep 25 at 21:04 | comment | added | COOLSerdash |
+1. To investigate whether the small sample bias of the robust standard errors are the culprit, you could redo the simulations using the dfadjust package, which promises to adjust for the bias.
|
|
| Sep 25 at 20:52 | history | answered | Lukas Lohse | CC BY-SA 4.0 |