Multivariate Regression vs T-test: and implication for multiple comparisons

Question

I have a data frame with different outcome measures (DV1:4) for participants some partcipant with additional IVs 1:2.

n=100;df <- data.frame(participant=1:n,DV1=rnorm(n),DV2=rnorm(n)
,DV3=rnorm(n),DV4=rnorm(n),IV1=rep(seq(1,2),n/2),IV1=rep(seq(2,1),n/2))

My initital thought was to use a t.test() for the different contrasts and run some post-hoc correction for multiple comparisons. I might opt for something more liberal than bonferoni correction ( What are Hommel Hochberg corrections?)

While investigating this correction I found that (Does one need to adjust for multiple comparisons when using MANOVA?) It's not necessary to correct for multiple comparisons in a multivariate model as it does it implicitly.

What would be the best option then?, to do multiple t tests or construct a multivariate model and do posthoc comparisons (of cause correcting for multiple comparisons)

I surpose that I could still control the direction of the alternative hypothesis by way of planned contrasts. And that there are different assumptions of the multivariate model to keep in mind.

Sorry if this has been asked before and Thanks for reading.

Answer 1

This is a tricky question. From your wording, I can't be sure that all participants have data on all of the IVs and DVs. Your pseudocode generates complete data, so I'm answering under that assumption.

With only four DVs, assuming you can put them in the same metric, you may want to consider a profile analysis (aka repeated measures MANOVA). You'd investigate 3 hypotheses: flatness (differences over DVs), levels (group differences), & parallelism (essentially the interaction of flatness & levels). On the right side of the equation, you could investigate the main effects and interaction between the IVs.

You can then do the post-hoc tests (perhaps use false dicovery rate instead of Bonferroni) you were mentioning.

Strictly speaking, your DVs don't have to be repeated for this analysis. You would just ignore the flatness hypothesis.

I've only performed profile analysis using SAS but it looks like you might able to do in in R with the profileR package.