I am struggling to understand should I report the results with random effects or not (re_formula = NULL or NA).

I have a population-based data. Patients are nested in differently sized counties and I would like to report the result closest to the truth. N = 6000. I am interested in population-level effect only (country level).

An additive non-hierarchical model shows a significant effect with the narrowest CIs.

Hierarchical free intercept model (1 | county) is equal according to loo(), but shows the same significant effect with slightly wider confidence intervals. R2 CIs are overlapping with the previous model, however, the mean is 2% lower for the hierarchical model.

The effects of the previous models will have much wider confidence intervals while I set “re_formula = NULL” in conditional_effects(). But they still show significant effects.

Which results should I report? Or how should I decide?

PS! I also did some frequential statistics. R2 including random effects was 9% better compared to fixed effects only.

  • $\begingroup$ I am not 100% on hierarchical models myself, but if patients are nested within county shouldn't the model have the random effect (1|county/patients)? Also I know for a fact though that overlapping CI's mean nothing. If you want to see if two parameters are equal you have to do some sort of test. Overlapping CI's don't mean they are statistically equal. $\endgroup$ – Kenney Jul 1 '20 at 22:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.