0
$\begingroup$

I asked participants to rate the valence of 92 objects on a 7-point scale from negative to positive. I want to know for each object whether it's valence is significantly different from 4, i.e., the neutral midpoint on the scale. I know that I'd run into multiple comparison problems if I did 92 one-sample t-tests. How do I best analyse this data? Do I correct for multiple comparisons applying a correction like Bonferroni? Or can I analyse this data using confidence intervals? Is there a multivariate solution? I read about the Hotelling's T2 test, but if I understand correctly, if it is significant that just tells me that at least one of the objects has a significantly positive or negative valence but not which ones nor whether it's positive or negative. Would I follow that up with univariate t-tests without correcting for multiple comparisons since the multivariate omnibus test was signficant? Any help would be much appreciated! Thanks in advance!

$\endgroup$

1 Answer 1

2
$\begingroup$

A pretty general approach to this is the closed testing principle. It tells you that you test the global null hypothesis (all =4), then sequences of intersection null hypotheses until you finally test the elementary null hypothesis. You cannot in general just go straight from rejecting the global null hypothesis to testing elementary null hypotheses.

Assuming that you truly need to control the type I error rate (rather than, say, the false discovery rate), the Bonferroni-Holm procedure is uniformly more powerful than Bonferroni (i.e. it always rejects a null hypothesis when Bonferroni does, but produces additional rejections).

If you need to control the type I error rate, then looking at confidence intervals does not really make any difference. Looking at whether two-sided 95% confidence intervals do not cover a null value (in your case 4) is equivalent (plus-minus some minor details in some specific situations where tests and confidence intervals are not completely equivalent) to do a two-sided null hypothesis test at the two-sided 5% level.

Whether you need to control the type I error rate is of course a different question that may depend on what you are trying to do/prove/whom you are trying to convince with your results.

$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.