Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.

Good day,
I'm working with a panel dataset, I've used many models, homogeneous (fixed effect, pooled ols and Driscoll and Kraay) heterogeneous (swamy random coefficients) and would like to do a post-estimation to select the model that best fit my regression. Is there any method, command that may allow me to do this? Any hint will be highly appreciated.
Ama

share|improve this question
1  
If you need a command to do that, you better provide us at least with the language/package you use. Are we talking SAS, SPSS, Stata, SPlus, Minitab, R, ... ? – Joris Meys Sep 14 '10 at 9:36

1 Answer

up vote 2 down vote accepted

According to my experience, model selection is not so much about seeking the model that best fits the regression. The first question to ask is : which model reflects my experimental design the best? The second question would then be : which model reflects the covariance structure in my data the best?

Only then you can start worrying about which model fits the data the best. There are a number of approaches to evaluate and comppare non-nested models. A naive approach would be to do a cross-validation and compare those results. One could eg do a leave-one-out crossvalidation, get the SS values for each run, and treat these SS values as sample data for model comparison.

There's a whole set of literature on comparing non-nested models, but that's always within the same framework. There's little possibilities for comparing models originating from different frameworks other than working with some general loss function.

share|improve this answer
Thanks a million for your valuable answer, it really does help, for your previous remark, i'm a stata user. – Ama Sep 14 '10 at 10:53
1  
In Stata, I wouldn't know a command to check that easily. When using implemented goodness of fit tests, you have to be aware you cannot compare the values across non-nested models in many cases, and definitely not across different frameworks. – Joris Meys Sep 14 '10 at 11:54
@Ama Maybe worth looking at the UCLA Resources Center for Stata, j.mp/9xBk6U – chl Sep 14 '10 at 12:14
Joris, Chl, many many thinks to both of you, your advices are really so valuable!!!! – Ama Sep 14 '10 at 14:40

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.