0
$\begingroup$

I'm not sure if this is answered or not because there's a lot of these questions and the ones I looked at didn't answer this.

If I were to say, have five independent variables of interest and one dependent variable, how would I adjust the p-value via Bonferonni (or another method) if there were 2 univariate analyses and 3 bivariate analyses and 1 multivariate analysis.

Let's say independent variables are A, B, C, D, and E and the dependent variable is Y. The analyses are as follows:

Univariates: Y ~ A, Y ~ B where A and B are continuous

Bivariate: Y ~ C + D, Y ~ C + E where C, D, E are binary

Multivariate: Y ~ A + C + D + E where A continuous, C, D, E are binary

Would I say, only do a correction on the p-value for two tests for the two univariate tests, three for the three bivariate, and no correction for the one multivariate test

OR

Correct the alpha boundary p-value for all of the tests since that's the number of total tests.

OR

Do I not adjust them at all? In class I was told to adjust them but for one of my big boy jobs, I was told to not adjust at all for a clinical trial.

$\endgroup$
  • $\begingroup$ Please clarify for us what "univariate," "bivariate," and "multivariate" analyses refer to in this multiple regression situation. Could you perhaps illustrate your meaning with a simple example? Conceivably the answers to your questions depend on the specific relationships among the hypotheses and data you are testing. $\endgroup$ – whuber Sep 9 '19 at 18:44
0
$\begingroup$

I remember reading, long ago (maybe 1990) a line from Jacob Cohen about the whole area of p value adjustment. He said "this is a question about which reasonable people can differ" (that might not be exact...but it's close).

I agreed then and I still do.

Much more recently, I read, in Statistics as Principled Argument by Robert Abelson (a marvelous book) that this sort of question is not so much about what can be done as what can be justified. So, if you don't adjust at all - can you justify that?

Also, remember that when you lower the risk of type I error you raise the risk of type II error and consider whether the two errors are equally bad in your analysis.

$\endgroup$
  • $\begingroup$ The part of the question that makes it different from similar ones on this site is its use of multiple models to assess a common dataset. The generalities in your answer don't address this issue. Could you speak to it specifically? $\endgroup$ – whuber Sep 10 '19 at 12:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.