3
$\begingroup$

I ran glm() for a binary outcome(0/1) not specifying family = binomial. So i just found that it ran as family= gaussian.

I already shared the odds ratios results with my clients and the odds ratios had a big difference..it seems went too far…..What is the best way to deal with this? Is there any way to deliver the real logistic regression model output and not really looking stupid like “i tried a different approach blah blah and found this!”

Please help!!

$\endgroup$
1
  • 5
    $\begingroup$ You've not modelled odds ratios but risk differences (badly), and presumably exponentiated those into meaninglessness. Either way the analysis is wrong and I think it can only hurt more not to be transparent about that. Your client doesn't need to know all the details, only that you'll make sure to provide the correct results -- statistical models are complicated after all. And perhaps you can have a think about avoiding issues like these in the future? $\endgroup$
    – PBulls
    Feb 24 at 6:44

3 Answers 3

5
$\begingroup$

I'm sure you have the sympathy of many people here; that's an unpleasant situation to find yourself in. It's also probably worth noting that some people might give different advice on a public internet site with names attached than if you talked to them late at night in a crowded bar.

That said, I think both professional ethics and consideration of the worst-case outcomes argue for admitting you made a mistake.

Consider:

"Inquisitive client: That's really interesting. Can you explain why you prefer the second analysis to the first one?"

or for a future consultant

"Yes, our previous statistician gave us a much better set of results but then they substituted this same disappointing one that you gave us. Why are statisticians so unhelpful?"

As an example of good practice, I nominate the National Mortality and Morbidity Air Pollution Study. They found out, in 2002, shortly after the regulatory deadline for new research inputs to the air pollution standards, that they had a mixture of wrong settings and software not quite fit for purpose. They admitted the problem; it's quite possible no-one would have noticed. The mistake ended up on the front page of the New York Times, which is probably a first for statistical computing. The statisticians involved, Francesca Dominici and Scott Zeger, were and are still regarded very highly in the statistical community. People are, by and large, a lot more forgiving of making mistakes in doing difficult things than they are of hiding them.

$\endgroup$
5
$\begingroup$

not really looking stupid like “i tried a different approach blah blah and found this!”

Your updated results would not be a different approach. They would reflect a correct approach. Or, at least, a much better approach - there is rarely a single unequivocally correct approach to modeling. Once you look at things this way, it becomes a case of rectifying an error, rather than offering an "alternative" after you have already delivered.

Yes, this is a very painful situation to be in. But you really owe this correction to your client, who paid you to work according to your professional ability. Leaving the incorrect results standing may lead to costly wrong decisions by your client. Follow-up work by your client, another statistician or even yourself might uncover the error, which may be even more embarrassing in a few weeks or months than being open about this now.

Your client may react negatively. Or they may be impressed by your professionalism. I am keeping my fingers crossed for the latter.

$\endgroup$
0
$\begingroup$

You ran a linear probability model. While there are issues with such a model, it is a model that exists in the statistics literature and has its advocates. Most of the advocacy I have seen for linear probability models can be summarized as, “I do not want to work with models beyond OLS linear regression,” but a linear probability model can be interpreted as a simple way to predict the probability of a binary outcome.

Because there is no link function that squashes the result to $[0, 1],$ the predicted probability values can exceed $1$ or fall below $0.$ How do you interpret this? Perhaps you begin to see the issues with the linear probability model.

I would encourage you to read about the pros and cons of linear probability models, but the ultimate test is if you are able to do useful work for your client. After all, if your linear probability model is able to give insights that reliably help your client, aren’t you doing your client a disservice by switching to a model that reliably helps them less?

(I have my doubts about your linear probability model compared to an analysis that isn’t rooted in forgetting to set a function argument (and I wonder if the R function should have a default family that allows this very mistake), however.)

What does strike me as shoddy work is running a linear probability model and then sending the results through the inverse link function of a logistic regression as if you had fit a logistic regression the whole time. If you know you’re delivering results based on a software mistake (neglecting to set a function argument), I think you owe the client a correction. I am a statistician, not an attorney, but I wonder if keeping this from your client could expose you litigation, either your firm or perhaps even you personally.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.