I've seen a couple of questions here around this alternative, but a -very basic- specific question I cannot answer from the top of my mind.

Having a metric measure $M$ taken before treatize and after treatize, I want to test the difference as the therapeutic effect in two groups, say material of therapeutic instrument is (k)eramic or (p)lastic.

The design is thought as first computing the difference $M_d=M_{after}-M_{before}$ and consider T-testing of $M_d$ over the two independent groups, call them $K$ and $P$.

But T-testing assumes normal distribution in the data, which might be violated or un-guaranteed when

  • a) $M_d$ is not normal,
  • b) $M_d$ in either group is not normal,
  • c) $N$ (or only $N_1$ or $N_2$) is too small to confirm normality.

But do I have to test before normality of $M_d$ the same test on the $M_{after}$ and $M_{before}$ in their subgroups?

The problem is on a couple of similar items and subgroups, and the subgroups have $N$ between, say 30 and 70.

I assume, that after only one of the conditions in a)...c) are not met, I have to use the Mann-Whitney-U-test on $M_d$.

Is this correct or is this overkill in testing at all? (I'm tempted to simply use U-test, because the $N$ are somehow in the near of the lower limit of size of subgroups.)

  • 1
    $\begingroup$ I wrote quite a bit about this here: stats.stackexchange.com/questions/538561/… $\endgroup$ Nov 22 '21 at 16:47
  • $\begingroup$ Christian - thanks for your link to your other answer, it is a good consideration. However I don't know how to use it in answering the sub-question whether it sufficies to test the normality of $M_d$ in the whole group or in the two subgroups separately, and as well with $M_{after}$ and $M_{before}$. In this hindsight it's also a technical question... $\endgroup$ Nov 22 '21 at 20:44
  • $\begingroup$ One implication of what I wrote is that testing is generally a somewhat deficient way to address the model assumptions issue, as it's not relevant whether data are "really normal" (they never are, for which reason normality can never be "confirmed"), but rather how exactly normality is violated, if it is violated in a way that would harm your conclusions (extreme outliers? strong skewness?). If this happens in even one group of interest, it's a problem. However there are deviations of normality that are harmless. Therefore better look at you data rather than testing. $\endgroup$ Nov 23 '21 at 0:09
  • $\begingroup$ By the way your sample sizes are solid, I'd say. Small sample sizes have two kinds of problems: Normality cannot be checked, and the U-test (which also under non-normality is not necessarily better than the t-test) loses a lot of power, so whatever you do will be bad, but this is not the situation in which you are. $\endgroup$ Nov 23 '21 at 0:13
  • $\begingroup$ See also arxiv.org/abs/1908.02218 $\endgroup$ Nov 23 '21 at 0:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.