2
$\begingroup$

I have multiple tests done on a similar subject (brain activity in different regions of the brain) I have a treatment group and control group. When comparing the two groups, on the 12 different brain regions, the treatment group had less brain activity in every region. However, because my sample size was 8 for treatment and 14 for control only two of them was significant. Is there something I can do to say that overall although 10 of the 12 regions didnt have significant it was likely because of low sample size for those and that there is a difference between the treatment and control overall? I know I could average all of the brain regions for each participant and do a t test on that but the sample size would still be low and this wouldnt be significant.

$\endgroup$
  • $\begingroup$ "I have multiple tests done on a similar subject"--by "subject" do you mean "topic" or "person/animal"? $\endgroup$ – AdamO Jun 28 '19 at 17:34
  • $\begingroup$ How did you perform the comparison? The most conservative comparison would be calculating a within-subject average and performing the 21 degree of freedom test. Your belief [that the test is anticonservative and findings are a false positive] may be right, but it also may be wrong. For instance, if you killed all 8 in the treatment group, their brain activity is 0 and the design of 8 vs 14 is very well powered. $\endgroup$ – AdamO Jun 28 '19 at 17:35
  • $\begingroup$ By multiple tests I mean topic. For example all 12 tests looked at brain activity but just different regions. I cant do within subjects because it is a between subjects design. I dont believe that the results may be false positive, I believe that there is an effect there, all brain regions had lower activity in the treatment condition. I was hoping there would be some kind of bulk analysis or something where I can analyze everything at once (all 12 brain regions in both the treatment and control participants) to increase the power. $\endgroup$ – Ryan Jun 28 '19 at 18:54
  • $\begingroup$ Not quite an answer to my question... but I gather "subject" means "person". So you did multiple tests for difference in hypothalamus, preoptic nucleus, subhypothalamic region... etc etc. using the same same sample of 8 vs 14. Correct? $\endgroup$ – AdamO Jun 28 '19 at 19:52
  • $\begingroup$ Subject means topic (brain activity) there were 8 people in the treatment and 14 in the control groups. It is a between subject design. 8 have beeen on the medication (treatment group), 14 have not been. Scans on 12 different brain regions were done in each individual. Thanks for all your help! $\endgroup$ – Ryan Jun 28 '19 at 20:28
0
$\begingroup$

The first thing to know is: what do you want to test ?

Is it: 1/ that the treatment has an effect on all areas of the brain ? 2/ or that the treatment has an effect on at least one area of the brain. 3/that the treatment has some kind of overall effect.

If it is 1/, it seems you cannot show it. Perhaps indeed due to sample size. Imagine the treatment has a true effect of 10 in the 2 areas where you find significant effects, but only a true effect of 3 in the other areas. With the same sample size, it is possible to find significant effects in the 2 areas, and no effect in the other 8 areas.

If it is 2/, you almost got it. You may have however to think to adjust your standard errors to "multiple hypotheses testing".

If it is 3/, why not indeed consider all observations in a same regression. In this case, I would add to the regression area-of-the-brain fixed-effects.

$\endgroup$
  • $\begingroup$ It is more 1 that I want to test. I want to see essentially if brain activity is slower in the treatment group than the control overall. All effects were in the same direction (slower in the treatment group) and the p values all under .2 with 2 of the regions having a p value of under .05. For example would it be possible to combine all of the data for each region? For example add all 12 regions together for all participants? This would give N = 8*12 = 96 for the treatment group and N = 12*14 = 168 for the control. $\endgroup$ – Ryan Jun 28 '19 at 19:12
  • $\begingroup$ I edited my post. I think you are more interested in 3/ than in 1/ (they differ). Please see above. Plus: out of curiosity, do you work on Stata ? $\endgroup$ – Alex. C-L - Reinstate Monica Jun 28 '19 at 23:12
  • $\begingroup$ No I don’t work on stata. Sorry I’m not kind of new to stats? What kind of regression are you referring to? And what would be the response and predictor variables? $\endgroup$ – Ryan Jun 29 '19 at 3:03
  • $\begingroup$ Area-of-the-brain fixed effect are dummy variables. For instance, for area A, it is 1 for all observations in area A, and 0 otherwise. So it implies adding 12 new variables. Or, more precisely, to estimate a "fixed-effect regression" with the id variable for the fixed-effect being the areas of the brain. Look at "fixed effect regression in your software on the internet or in your software". I hope it helps. $\endgroup$ – Alex. C-L - Reinstate Monica Jun 29 '19 at 5:42
  • $\begingroup$ Note: if you follow such path, you will have to think in a second time to eventual clustering of the standard error per individual or area of the brain I think (it usually increases them a lot). But the design above is not something I already used as such, so I do not know exactly what to say here. $\endgroup$ – Alex. C-L - Reinstate Monica Jun 29 '19 at 6:02

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.