Timeline for How can I compare models without fitting?
Current License: CC BY-SA 4.0
6 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 15, 2019 at 18:51 | vote | accept | jbacks | ||
| Feb 15, 2019 at 18:51 | comment | added | jbacks | This seems plausible, and I'm going to give it a shot. Thank you again for your insight. | |
| Feb 14, 2019 at 20:52 | comment | added | EdM | @jbacks also do consider working with observations/predictions in a transformed scale such as logarithmic. A bias in error terms in a non-transformed scale might be reduced or removed following transformation. Note that the use of percent errors, suggested in another answer, is equivalent to looking at differences between log-transformed predictions and observations. You'll have to judge whether that would be appropriate for this situation. | |
| Feb 14, 2019 at 20:45 | comment | added | EdM | @jbacks I don't know of a published precedent but I don't think this approach would be a hard sell if there is a solid theoretical basis for your model(s). In this theory-based analysis, focus on the reasons for the systematic bias (non-zero mean error, Issue I) between predictions and observations. That would seem to get most directly at the relative worth of the models. Issue II (any patterns of error magnitude/direction related to independent variable values or predicted values) should illustrate where your models are going astray. Comparisons of model variances are of less interest. | |
| Feb 14, 2019 at 20:26 | comment | added | jbacks | Thinking of the residual distribution as the object of comparison is a useful shift in perspective! a) Would you know of any published analyses that use a similar method? I feel like my situation is unusual. Any published precedent would be helpful. b) The mean of each residual distribution is non-zero and visibly different for two of my models, and I expect ANOVA would confirm this. Knowing this, would it still be sensible to examine the differences among the variance of each residual distribution (Issue 3)? Could patterns exposed via Issue 2 invalidate a comparison of variances? | |
| Feb 13, 2019 at 23:59 | history | answered | EdM | CC BY-SA 4.0 |