I have recently made a simulation program which simulates patients with Type 1 diabetes. In that context I am creating artifical patients. Let's denote three of these as $\text{p}_1$, $\text{p}_2$, and $\text{p}_3$. Assume that $\text{p}_1$, $\text{p}_2$ and $\text{p}_3$ are treated with treatment $A$. Treatment $A$ treats $\text{p}_1$, $\text{p}_2$ and $\text{p}_3$'s diabetes to a certain degree. We evaluate the treatment with some measures conducted in a virtual trial; let's denote one of these measures $\text{M}1_A$. $\text{M}1_A$ has an average value and a standard deviation for Treatment A for the three patients.

Now let's introduce treatment B for the same patient group, $\text{p}_1$, $\text{p}_2$ and $\text{p}_3$. Again we put the treatment through a trial test and receive a list of measures, this time denoted $\text{M}1_B$. I am interested in testing whether the difference in $\text{M}1_A$ and $\text{M}1_B$ is significant.

I know: $N=3, \text{std(M}1_A), \text{std(M}1_B), \text{mean(M}1_A), \text{mean(M}1_B)$


I want to compare the two measures tested on the same group of 'people' and use a statistical tool to figure out if the difference in the two measures are significant.

I am considering using Welch's t-test – would this be correct?

  • $\begingroup$ WHen you say you know the means and standard deviations, do you not have the three original values for each treatment? $\endgroup$ – Glen_b Mar 18 '14 at 8:46
  • $\begingroup$ I conduct the virtual trial on each patient for both treatments. This means that I can calculate the mean and standard deviation for the group on treatment A and the mean and standard deviation for the group on treatment B. @Glen_b I am not sure what you mean by original but, I would assume that yes i have the original values. $\endgroup$ – SteewDK Mar 18 '14 at 8:51
  • $\begingroup$ If you're simulating, what makes the the results "on the same patient" rather than "on different patients" (i.e. what makes the same-patient measures 'more alike')? $\endgroup$ – Glen_b Mar 18 '14 at 9:24
  • $\begingroup$ I am simulating so this means that treatment A will not effect treatment B. I assume that the treatments are done on the EXACT same patients - nothing differs between the two groups execpt the treatment itself. Thanks @Glen_b. $\endgroup$ – SteewDK Mar 18 '14 at 9:50
  • $\begingroup$ If there's no effective difference in the simulation if you'd used patients p4,p5,p6 for the second treatment in your simulation, then it's not actually paired -- but if you do a real experiment, it matters a lot. $\endgroup$ – Glen_b Mar 18 '14 at 10:04

If they are the same three patients, the measurements are paired, in which case a Welch t-test is not ideal. A paired test of some kind, perhaps a paired t-test, a permutation test (that deals with the pairing), or a Wilcoxon signed rank test might be suitable, perhaps.

(You might also need to worry about order effects in your design.)

  • $\begingroup$ Thanks for your reply. What would you use yourself? I am not that much into statistics so what exactly do you mean when you state that I should worry about order effects? How should I handle this? $\endgroup$ – SteewDK Mar 18 '14 at 8:14
  • $\begingroup$ I don't know the circumstances; in some circumstances I might use any of those, depending on what exactly is the question of interest, what I know or feel safe assuming, or how badly the assumptions of the various procedures are likely to be wrong and how sensitive they are to those assumptions. In other circumstances I might do other things than those three. $\endgroup$ – Glen_b Mar 18 '14 at 8:41
  • $\begingroup$ If you do an actual test of two treatments on a single patient, impact on the effect of the second treatment caused by being exposed to the first treatment before it would either be an order effect or a carry-over effect (which also matters). (ctd) $\endgroup$ – Glen_b Mar 18 '14 at 9:28
  • $\begingroup$ (ctd) ... (Note that a paired design is a repeated measures design with only two measures.) -- the way to deal with order effects in practice is to balance the order of treatment (randomize patients half to one treatment first, half to the other treatment first). $\endgroup$ – Glen_b Mar 18 '14 at 9:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.