Is there ever a compelling reason to conduct multiple hypothesis tests versus a multiple regression when comparing a response in two treatment groups among many (possibly thousands or millions) explanatory variables?

For example, say we have the alternatives:

  • Multiple comparisons of the response $y$ in treatment group $x_{trmt}=A$ vs. $x_{trmt}=B$ on many variables $x_1,x_2,...,x_p$ (controlling for the familywise error rate or false discovery rate with Bonferroni, Benjamini–Hochberg procedure, etc.), or
  • Multiple regression of $y$ on $x_{trmt},x_1,x_2,...,x_p$ with interaction terms $x_{trmt}x_1,x_{trmt}x_2,...,x_{trmt}x_p$.

Especially when considering strong positive or negative correlations between the explanatory variables may exist, a penalized regression such as a lasso or ridge regression would seem a more powerful alternative to multiple comparison techniques that avoids issues with interpreting $p$-values. The lasso regression can also handle data where the number of explanatory variables exceeds the number of subjects in the dataset.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.