0
$\begingroup$

I performed a randomization test and found that the means of groups A and B are significantly different. I am trying to prove that group B is "special", that there is something interesting going on and its mean is different than the background.

In a second test, would it be problematic if I compared the difference in means between groups B and (A+B) i. e. would it be a problem that group B is contained in group (A+B), that I'm testing it against? Or would the results of this test simply not bring any value, since I've already found that the means of A and B are different?

$\endgroup$
  • $\begingroup$ If you want to assess the difference between A and B, then don't look at the difference between A and A&B together. What do you mean by 'background' and do you have any data on that? $\endgroup$ – BruceET Jul 17 at 2:10
  • $\begingroup$ I have a multivariate time series of chemical signals. Groups A and B are all the samples. B are the samples that correspond to volcanic eruptions. I am trying to show that the levels of a certain chemical are significantly higher in group B i.e. in samples associated with volcanic eruptions. This is why I was thinking that maybe it would make sense to compare the mean of samples associated with volcanic eruptions (group B) with the mean of all samples (groups A and B). $\endgroup$ – Marius Orehovschi Jul 17 at 2:31
0
$\begingroup$

The purpose seems to be to provide evidence that population B is different than population A. You use as vehicle the difference in expected values. Translating that in terms of hypothesis testing means that the null hypothesis is that you have a single population against the alternative hypothesis that the populations are different. This looks ok to me. However A against A+B implies a mix of populations, if what you try to prove is correct. This looks wrong because the estimated expected value of the mix depends on sample sizes, and even for same sample sizes you loose a lot of power. Also tests like t test assumes a simple normal distribution which often is not the case for a mix of populations. For example a mix of gaussians is often bimodal if variances are small enough.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.