I am analyzing adverse events rate in two groups, treatment and control group. There are about 10 different adverse events. Most of the rates are not significantly different between these two groups, however, the rates are systematically higher in the treatment group. What does that mean? Is it related to sample size?
$\begingroup$
$\endgroup$
1
-
$\begingroup$ It might be meaningless. The key is to assess the degree to which these ten events might be correlated. If you can argue they are independent, then you can "chunk" them and perform an "omnibus" test using an F-ratio statistic and almost surely it would have a very tiny p-value, reflecting the consistency of ten independent events. It would help to know (much) more about what an "adverse event" means and how they were measured or observed. What is the experimental design? $\endgroup$– whuber ♦Commented Apr 21, 2023 at 15:07
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
7
The power of the test is based on sample size yes, but more importantly on the amount of events. All proportions of events are very low, (e.g., 0.35% of 4000 = 0.0035 * 4000 = 14 events. This considerably limits the power of your statistical test. Therefore, there is much uncertainty whether these differences are truly systematic.
-
$\begingroup$ Thanks a lot for your input. Then is there any better way to do the test? $\endgroup$– genomyCommented Apr 21, 2023 at 12:20
-
$\begingroup$ Not really, it seems as though this analysis is just underpowered. You could look at the total count of adverse events by adding all of them up, just to see if the total amount of adverse events (regardless of type) differs between groups (at which point you have more events to power a test), but this might not be your research question. $\endgroup$– rjjanseCommented Apr 21, 2023 at 12:21
-
$\begingroup$ You are right, the sum of adverse events is not really what I am interested in. Thank you. $\endgroup$– genomyCommented Apr 21, 2023 at 14:47
-
-
$\begingroup$ I think much more information about the question needs to be disclosed before this or any answer is accepted. $\endgroup$– whuber ♦Commented Apr 21, 2023 at 15:08