Timeline for Explanation of minimum observations for multiple regression
Current License: CC BY-SA 3.0
11 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Apr 16, 2013 at 1:23 | answer | added | playitagain | timeline score: 1 | |
| S Apr 15, 2013 at 23:13 | history | suggested | sashkello | CC BY-SA 3.0 |
formulas formatting
|
| Apr 15, 2013 at 22:59 | review | Suggested edits | |||
| S Apr 15, 2013 at 23:13 | |||||
| Apr 15, 2013 at 15:11 | comment | added | cryptic_star | whuber, that last bit about choosing wisely among so many models when you have a fewer number of observations is the crux of the matter, and I'm still pondering it. ^_^ It still seems to me though that forward stepwise regression allows for getting closer to being able to make that choice with a smaller amount of data, although Scortchi's response below indicates that I may not have the most sound methodology with that thought. | |
| Apr 15, 2013 at 13:42 | comment | added | whuber♦ | Re (1) That's not "certain"--that's an estimate based on your data. You are, in effect, using your data for double duty: once to test whether the relationships appear linear, then again to fit the relationships. When you have only $N-1$ predictors, though, there is no way you can evaluate linearity. That already points to a need for some extra data. Re (2), perhaps your question ought to be "how is it even possible to choose wisely among $2^{p+1}-1$ models if I have fewer than $p + 2^{p+1}$ independent observations?" :-) | |
| Apr 15, 2013 at 13:37 | comment | added | cryptic_star | (1) Yes, we are certain that the relationship is linear - we've been sure to do some trend analysis before starting the regression to see if we need any transformations. (2) We are not certain about our independent variables though. We are planning on performing some stepwise regression to learn more about the relationships that are present. | |
| Apr 15, 2013 at 13:03 | history | edited | user88 | CC BY-SA 3.0 |
edited title
|
| Apr 15, 2013 at 13:02 | comment | added | whuber♦ | +1 I think this is a great question: it cogently challenges accepted wisdom, forcing us to think harder about why certain practices are correct and good. But, to narrow the scope a little, I would like to ask about your situation. (1) Are you certain the relationship between the dependent and all independent variables is linear? (2) Are you certain of your independent variables--that is, should they all be included or not? After all, you mention "overfitting": bear in mind, then, that with $p$ possible variables you have $2^{p+1}-1$ possible models to fit. That might require a lot of data! | |
| Apr 15, 2013 at 12:46 | answer | added | Scortchi♦ | timeline score: 1 | |
| Apr 15, 2013 at 11:54 | answer | added | sashkello | timeline score: 3 | |
| Apr 15, 2013 at 11:28 | history | asked | cryptic_star | CC BY-SA 3.0 |