0
$\begingroup$

I have a study in which we are looking at healthcare costs and utilization by patients with a disease vs. healthy controls (HC), as well as by different stratifications of disease (severity, comorbidities).

Summary of all comparisons are below:

Part 1. Healthcare costs

  • Inpatient: 1) Disease vs. HC, 2) mild Disease vs. HC, 3) severe Disease vs. HC, 4) Disease + 1 comorbidity vs. Disease alone, 5) Disease + 2 comorbidities vs. Disease alone, and 6) Disease + 3 comorbidities vs. Disease alone
  • Outpatient: same as above in inpatient
  • Emergency room: same comparisons as above
  • Pharmacy costs: same comparisons as above

Part 2. Healthcare utilizations

  • Disease vs. HC in odds of seeing 15 different types of specialty providers
  • Mild disease vs. HC in odds of seeing 15 different types of specialty providers
  • Severe disease vs. HC in odds of seeing 15 different types of specialty providers
  • Disease + 1 comorbidity vs. Disease alone in seeing 15 different types of specialty providers
  • Disease + 2 comorbiditeis vs. Disease alone in seeing 15 different types of specialty providers
  • Disease + 3 comorbidtiies vs. Disease alone in seeing 15 different types of specialty providers

If I were to use Bonferroni for multiple comparison correction, can I use the different categories to have separate Bonferroni corrections for each one?

In other words, for Part 1, can my Bonferroni p-value cut off be: 0.05/6 = 0.0083 for each of Inpatient, Outpatient, Emergency, and Pharmacy category? Or will it be 0.05/24 =0.002 lumping all the comparisons in the different categories together?

Similarly, for Part 2, can my Bonferroni p-value cut off be: 0.05/15 = 0.003 for each of the comparisons? Or will it be 0.05/90 = 0.0005??

Thanks in advance!

$\endgroup$
1
  • 1
    $\begingroup$ See stats.stackexchange.com/questions/120362/… for arguments against using the Bonferroni correction at all. The only good reason to use it is if people who have the power to stop you publishing want you to. If those people are ok with using categories in the way you suggest, go ahead. $\endgroup$
    – fblundun
    Apr 2 at 19:03
0
$\begingroup$

Bonferroni and other multiple comparisons procedures are quite useful to keep researchers honest. It is easy to cherry-pick statistical results to get whatever conclusions you want, simply by taking advantage of randomness.

Sure, you can use separate groups, and this is commonly done, eg, in pharmaceutical studies. But such a data analysis plan must be pre-specified, otherwise, you run the risk of the same cherry-picking mistakes that Bonferroni and other multiple comparisons procedures aim to avoid. You also need to acknowledge that the maximum overall familywise type I error rate (FWER) is 0.10, not 0.05, when you have two separate families.

There are improvements that can be made over simple Bonferroni by incorporating dependencies, logical constraints, distributional characteristics (eg discreteness), and hierarchical structures, while still controlling the FWER.

You also might want to consider whether you would be satisfied by controlling the false discovery rate (FDR) instead of the FWER. You might just lump the entire set of comparisons into a single family in that case.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.