I have a logistic regression model that is seemingly significant when regressing individual variables in a univariate regression, but the entire thing falls apart when input into a multiple model. I've tried my best to fix multicollinearity problems.

The thing I'm most surprised is how BMI goes from being very significant to non-significant, as there is a strong clinical suspicion that BMI should play a part and it seems to do so when univariately regressed.

EDITED data file out as question has been answered.


The thing that strikes me about your example data is the low proportion (about 4%) of Fractures. That may very well be legitimate, but it does mean your data contains very little information. In the example data there are only 25 persons with a fracture. With such a tiny sample size it is no surprise that the model becomes highly unstable, especially in a multivariate model.

I suspect your data is just not useful for that research question. This is obviously not very nice but at some point it is just good to remember the following quote from John Tukey (1986, p.74-75):

The combination of some data and an aching desire for an answer does not ensure that a reasonable answer can be extracted from a given body of data.

A possible design that could be more promising is a case control study.

John Tukey (1986), "Sunset salvo". The American Statistician 40(1):72-76. https://doi.org/10.1080/00031305.1986.10475361

  • $\begingroup$ Thank you Maarten, would you suggest not running a multiple regression and stick to association models instead to answer the research question to the best of the data's ability? $\endgroup$ – Paze Jan 8 at 13:05
  • $\begingroup$ Or maybe extract 25 non fractures at random and do a case control instead like you said? Would that be better? $\endgroup$ – Paze Jan 8 at 13:09
  • $\begingroup$ Perhaps instead of continuous data we could dichotomize the data as well? $\endgroup$ – Paze Jan 8 at 13:17
  • $\begingroup$ I don't think the data can be used at all for your research problem. There is just not enough information present. If that is true, then you need to throw away the data you have (at least for this project) and start over with collecting new data using a different design, like case control, that can handle such sparse events. $\endgroup$ – Maarten Buis Jan 8 at 13:34
  • $\begingroup$ The point of a case control study is that you can use targeted sampling techniques to sample a large number from the rare group, and separately sample from the control population. This way the rare group is no longer rare in your sample. So extracting the 25 people with fractures from your data is not going to work, as this way you don't increase the number of people with fractures. To use a case control study you need to forget about the data you have and start the data collection all over again. $\endgroup$ – Maarten Buis Jan 8 at 13:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.