Suppose we have a regression model. If we get estimated of some of the coefficients and the standard errors are high, does this mean that the model is wrong/bad? How exactly do statistical packages choose regression models (in particular ordinal regression)?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ What do you mean by "How exactly do statistical packages choose regression models (in particular ordinal regression)?"? Are you asking how the models are fit? $\endgroup$– MacroCommented Jan 9, 2013 at 13:36
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
6
The "goodness" or "badness" of a regression model cannot be judged by any set of statistics alone. A model is "good" if it enlightens you, helps you solve a problem.... etc. or, to the extent to which it meets the "Magic" criteria, as introduced by Robert Abelson in his book Statistics as Principled Argument (link goes to my review of the book).
A high standard error (relative to the coefficient) means either that 1) The coefficient is close to 0 or 2) The coefficient is not well estimated or some combination. "High" by itself doesn't really have a set meaning (you can change the SE by changing the unit - measure in miles instead of microns and the SE will be tiny).
answered Jan 8, 2013 at 19:23
-
$\begingroup$ I would not say that a high standard error means that "the coefficient is close to 0". Proximity to 0, & the size of the SE are conceptually unrelated. Perhaps you're thinking of high p-value. $\endgroup$ Commented Jan 9, 2013 at 0:06
-
1$\begingroup$ No, I was saying "relative to the coefficient" this is true. You can, of course, have a high SE and a high coefficient, that's why my 1) is only one of two possibilities. $\endgroup$ Commented Jan 9, 2013 at 0:20
-
2$\begingroup$ Hmm, that's a good point (I should read more carefully). However, I can't tell if the OP means that their SE's are high relative to the coefficients, or just high in general; the question seems ambiguous on this point. $\endgroup$ Commented Jan 9, 2013 at 0:44
-
1$\begingroup$ Peter, It seems to me that whether or not large standard errors indicate a problem depends on the situation. E.g. if the sample size or effect sizes are small, then it's just a fact of life but, if there's large collinearity, you should probably do something about it before "going to town" interpreting the output of your model. I don't disagree with your answer but it presupposes no background info, making it not particularly useful, like fitting a model with no predictors. $\endgroup$– MacroCommented Jan 9, 2013 at 13:51
-
2$\begingroup$ I agree with you and @gung that more clarification is needed. In fact, I think this question/answer (and others like it) may benefit from some of your own advice. There's no need to treat questions like these as missing data problems :) $\endgroup$– MacroCommented Jan 9, 2013 at 13:58