3
$\begingroup$

My binary logistic regression gives P values significant at 0.05 level for some of the modeled independent variables. However, bootstrapping the same regression model gives P values significant at 0.01 or 0.001 levels for the same predictors which were mostly significant only at 0.05 level, or even for some of them which were not significant!

Does it have a worthwhile meaning? A huge discrepancy between the two P values of the same variable worries me, especially when I see the bootstrapped one that I expected to give more conservative results is giving much stronger results! I read somewhere that in such a case, we should discuss that the results pertaining to these variables should be assessed further. But there were no explanations and such a recommendation doesn't seem so specific.

Which P value is better here? Are both correct? (especially the bootstrapped one).

Do you have any recommendations for this strange P value behavior?

$\endgroup$
  • 2
    $\begingroup$ If bootstrapped and parametric ps differ a lot, my first guess would be that the assumptions of the parametric model somehow are violated. So I would check for that. I would not agree that the boostrap generally gives more conservative results. $\endgroup$ – Felix S Jul 10 '13 at 8:21
  • $\begingroup$ Many thanks dear Felix. Then the bootstrapped one is the more reliable one. $\endgroup$ – Vic Jul 10 '13 at 17:28
  • $\begingroup$ Dear Felix I looked up logistic regression's assumptions and see none of them are violated. So do you have in mind any other reasons for the huge biases and differences between the P values calculated by the non-bootstrapped vs. bootstrapped models? Thanks again. $\endgroup$ – Vic Aug 2 '13 at 18:25
1
$\begingroup$

If bootstrapped and parametric ps differ a lot, my first guess would be that the assumptions of the parametric model somehow are violated. So I would check for that. I would not agree that the boostrap generally gives more conservative results.

$\endgroup$
  • $\begingroup$ I've copied this comment by @FelixS as a community wiki answer because the comment is, more or less, an answer to this question. We have a dramatic gap between answers and questions. At least part of the problem is that some questions are answered in comments: if comments which answered the question were answers instead, we would have fewer unanswered questions. $\endgroup$ – mkt - Reinstate Monica Aug 13 '18 at 14:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.