Take the 2-minute tour ×
Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. It's 100% free, no registration required.

I am reflecting on the discussion around this question and particularly Frank Harrell's comment that the estimate for variance in a reduced model (ie one from which a number of explanatory variables have been tested and rejected) should use Ye's Generalized Degrees of Freedom. Professor Harrell points out this will be much closer to the residual degrees of freedom of the original "full" model (with all the variables in) than that from a final model (from which a number of variables have been rejected).

Question 1. If I want to use an appropriate approach to all the standard summaries and statistics from a reduced model (but short of a full implementation of Generalized Degrees of Freedom), would a reasonable approach be to just use the residual degrees of freedom from the full model in my estimates of residual variance, etc?

Question 2. If the above is true and I want to do it in R, might it be as simple as setting

finalModel$df.residual <- fullModel$df.residual

at some point in the model fitting exercise, where finalModel and fullModel were created with lm() or a similar function. After which functions such as summary() and confint() seem to work with the desired df.residual, albeit returning an error message that someone has clearly mucked around with the finalModel object.

Thanks.

share|improve this question
8  
Good question. This is related to why Douglas Bates doesn't include p-values in lmer output. See his reasoning here. –  Mike Wierzbicki Feb 14 '12 at 15:43
1  
Thanks - great link. –  Peter Ellis Feb 14 '12 at 18:18
    
I have seen full model df used in such a situation more than once. (Ye's approach comes up a lot in different situations; it's a paper I find myself recommending to people regularly. It would be nice to have some generic-but-efficient R function that many functions could take advantage of.) –  Glen_b Nov 29 '12 at 2:06

1 Answer 1

Do you disagree with @FrankHarrel's answer that parsimony comes with some ugly scientific trade-offs, anyways?

I love the link provided in @MikeWiezbicki's comment to Doug Bates' rationale. If someone disagrees with your analysis, they can do it their way, and this is a fun way to start a scientific discussion about your base assumptions. A p-value does not make your conclusion an "absolute truth".

If the decision of whether or not to include a parameter in your model comes down to "picking hairs" over what are, for scientifically meaningful samples, relatively small discrepancies in the df -- and you are not dealing with $n<p$ problems that justify more nuanced inference, anyways -- then you have a param so close to meeting your cutoffs that you should be transparent and talk about it either way: just include it, or analyze the model with and without it, but definitely transparently discuss your decision in the final analysis.

share|improve this answer
    
+1 and I am now inclined to agree that in fact my original question is not that important given these other issues –  Peter Ellis Mar 1 '13 at 22:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.