I have multiple tests done on a similar subject (brain activity in different regions of the brain) I have a treatment group and control group. When comparing the two groups, on the 12 different brain regions, the treatment group had less brain activity in every region. However, because my sample size was 8 for treatment and 14 for control only two of them was significant. Is there something I can do to say that overall although 10 of the 12 regions didnt have significant it was likely because of low sample size for those and that there is a difference between the treatment and control overall? I know I could average all of the brain regions for each participant and do a t test on that but the sample size would still be low and this wouldnt be significant.
6
-
$\begingroup$ "I have multiple tests done on a similar subject"--by "subject" do you mean "topic" or "person/animal"? $\endgroup$– AdamOCommented Jun 28, 2019 at 17:34
-
$\begingroup$ How did you perform the comparison? The most conservative comparison would be calculating a within-subject average and performing the 21 degree of freedom test. Your belief [that the test is anticonservative and findings are a false positive] may be right, but it also may be wrong. For instance, if you killed all 8 in the treatment group, their brain activity is 0 and the design of 8 vs 14 is very well powered. $\endgroup$– AdamOCommented Jun 28, 2019 at 17:35
-
$\begingroup$ By multiple tests I mean topic. For example all 12 tests looked at brain activity but just different regions. I cant do within subjects because it is a between subjects design. I dont believe that the results may be false positive, I believe that there is an effect there, all brain regions had lower activity in the treatment condition. I was hoping there would be some kind of bulk analysis or something where I can analyze everything at once (all 12 brain regions in both the treatment and control participants) to increase the power. $\endgroup$– RyanCommented Jun 28, 2019 at 18:54
-
$\begingroup$ Not quite an answer to my question... but I gather "subject" means "person". So you did multiple tests for difference in hypothalamus, preoptic nucleus, subhypothalamic region... etc etc. using the same same sample of 8 vs 14. Correct? $\endgroup$– AdamOCommented Jun 28, 2019 at 19:52
-
$\begingroup$ Subject means topic (brain activity) there were 8 people in the treatment and 14 in the control groups. It is a between subject design. 8 have beeen on the medication (treatment group), 14 have not been. Scans on 12 different brain regions were done in each individual. Thanks for all your help! $\endgroup$– RyanCommented Jun 28, 2019 at 20:28
|
Show 1 more comment
1 Answer
$\begingroup$
$\endgroup$
5
The first thing to know is: what do you want to test ?
Is it: 1/ that the treatment has an effect on all areas of the brain ? 2/ or that the treatment has an effect on at least one area of the brain. 3/that the treatment has some kind of overall effect.
If it is 1/, it seems you cannot show it. Perhaps indeed due to sample size. Imagine the treatment has a true effect of 10 in the 2 areas where you find significant effects, but only a true effect of 3 in the other areas. With the same sample size, it is possible to find significant effects in the 2 areas, and no effect in the other 8 areas.
If it is 2/, you almost got it. You may have however to think to adjust your standard errors to "multiple hypotheses testing".
If it is 3/, why not indeed consider all observations in a same regression. In this case, I would add to the regression area-of-the-brain fixed-effects.
-
$\begingroup$ It is more 1 that I want to test. I want to see essentially if brain activity is slower in the treatment group than the control overall. All effects were in the same direction (slower in the treatment group) and the p values all under .2 with 2 of the regions having a p value of under .05. For example would it be possible to combine all of the data for each region? For example add all 12 regions together for all participants? This would give N = 8*12 = 96 for the treatment group and N = 12*14 = 168 for the control. $\endgroup$– RyanCommented Jun 28, 2019 at 19:12
-
$\begingroup$ I edited my post. I think you are more interested in 3/ than in 1/ (they differ). Please see above. Plus: out of curiosity, do you work on Stata ? $\endgroup$ Commented Jun 28, 2019 at 23:12
-
$\begingroup$ No I don’t work on stata. Sorry I’m not kind of new to stats? What kind of regression are you referring to? And what would be the response and predictor variables? $\endgroup$– RyanCommented Jun 29, 2019 at 3:03
-
$\begingroup$ Area-of-the-brain fixed effect are dummy variables. For instance, for area A, it is 1 for all observations in area A, and 0 otherwise. So it implies adding 12 new variables. Or, more precisely, to estimate a "fixed-effect regression" with the id variable for the fixed-effect being the areas of the brain. Look at "fixed effect regression in your software on the internet or in your software". I hope it helps. $\endgroup$ Commented Jun 29, 2019 at 5:42
-
$\begingroup$ Note: if you follow such path, you will have to think in a second time to eventual clustering of the standard error per individual or area of the brain I think (it usually increases them a lot). But the design above is not something I already used as such, so I do not know exactly what to say here. $\endgroup$ Commented Jun 29, 2019 at 6:02