Suppose we have a regression model. If we get estimated of some of the coefficients and the standard errors are high, does this mean that the model is wrong/bad? How exactly do statistical packages choose regression models (in particular ordinal regression)?

  • $\begingroup$ What do you mean by "How exactly do statistical packages choose regression models (in particular ordinal regression)?"? Are you asking how the models are fit? $\endgroup$ – Macro Jan 9 '13 at 13:36

The "goodness" or "badness" of a regression model cannot be judged by any set of statistics alone. A model is "good" if it enlightens you, helps you solve a problem.... etc. or, to the extent to which it meets the "Magic" criteria, as introduced by Robert Abelson in his book Statistics as Principled Argument (link goes to my review of the book).

A high standard error (relative to the coefficient) means either that 1) The coefficient is close to 0 or 2) The coefficient is not well estimated or some combination. "High" by itself doesn't really have a set meaning (you can change the SE by changing the unit - measure in miles instead of microns and the SE will be tiny).

  • $\begingroup$ I would not say that a high standard error means that "the coefficient is close to 0". Proximity to 0, & the size of the SE are conceptually unrelated. Perhaps you're thinking of high p-value. $\endgroup$ – gung Jan 9 '13 at 0:06
  • 1
    $\begingroup$ No, I was saying "relative to the coefficient" this is true. You can, of course, have a high SE and a high coefficient, that's why my 1) is only one of two possibilities. $\endgroup$ – Peter Flom Jan 9 '13 at 0:20
  • 2
    $\begingroup$ Hmm, that's a good point (I should read more carefully). However, I can't tell if the OP means that their SE's are high relative to the coefficients, or just high in general; the question seems ambiguous on this point. $\endgroup$ – gung Jan 9 '13 at 0:44
  • $\begingroup$ I agree. This question (as so many) is ambiguous $\endgroup$ – Peter Flom Jan 9 '13 at 0:52
  • 2
    $\begingroup$ I agree with you and @gung that more clarification is needed. In fact, I think this question/answer (and others like it) may benefit from some of your own advice. There's no need to treat questions like these as missing data problems :) $\endgroup$ – Macro Jan 9 '13 at 13:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.