3
$\begingroup$

Given a trial to test a new drug. There are 20 volunteers with some disease. The volunteers are split into two groups (10 volunteers in each) randomly: the treatment group and the control group. Volunteers in the treatment group receive the new drug, volunteers in the control group receive placebo (pills that looks like a drug but do not have active substance). You conclude that new drug is effective if people who take the drug will recover faster (on average) than people in the control group. If your drug is effective, you will invest in its production, otherwise you will look for another drug. Assume that you obtained the following data (disease duration in days)

c​ontrol group treatment group
6 7
7 6
7 6
5 5
7 5
8 6
8 7
7 5
7 5
7 8

Would you invest into production of this drug?

For the above problem formulation, I am using 2 sample unpaired t test with equal variance and the following hypothesis:

$$H_0: \mu_{X} = \mu_{Y}$$ $$H_1: \mu_{X} > \mu_{Y}$$

where the population parameters $\mu_{X}$ and $\mu_{Y}$ representing average recovery duration for control and treatment group. Thus when the significance level is 0.05, the null hypothesis can be rejected in favor of alternate hypothesis after getting the t-statistic and p value according to the test.

Could you please confirm if my approach to the above problem formulation is correct? Also I have taken equal variances as sample sizes are same and are independent and the sample standard deviation is approximately same and the alternative hypothesis as upper one tailed.

$\endgroup$
2
  • 3
    $\begingroup$ Perhaps the most important aspect of this situation is your consideration of the possibility the treatment will be worse than the control. This will influence your choice of $H_1$ and will affect the decision. A good answer, then, would explain your choices: why these hypotheses and why this particular form of the t test? $\endgroup$
    – whuber
    Jun 5 at 19:24
  • $\begingroup$ @whuber thank you then for the alternate hypothesis, should I consider μX<μY and calculate the t statistic and p value on it? $\endgroup$
    – ten do
    Jun 5 at 20:39
2
$\begingroup$

I have put your data into R and looked at numerical and graphical summaries.

ctrl = c(6,7,7, 5,7,8,8, 7,7,7)
trtm = c(7,6,6, 5,5,6,7, 5,5,8)

summary(ctrl); sd(ctrl)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
    5.0     7.0     7.0     6.9     7.0     8.0 
[1] 0.875595  # sample SD
summary(trtm); sd(trtm)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
   5.00    5.00    6.00    6.00    6.75    8.00 
[1] 1.054093

stripchart(list(ctrl,trtm), meth="stack", ylim=c(.5,2.7), pch=20)

enter image description here

First impressions are the Treatment counts may tend to be smaller than Control counts. However, the average improvement is a bit less than one day for a malady that seems to last about a week without treatment. So, at the levels of drug administered in this small study the effect is not huge even if it is real.

Also, I am always skeptical when equal variances are assumed without a good prior rationale. Unless, you are somehow sure ahead of time that the treatment can do no harm, I would do a two-sided test.

A two sided Welch two-sample t test is significant at the 10% level, but not at the 5% level. So I would say the data suggest the Treatment may be effective. These are very small groups for a clinical trial, so I am not surprised that there is only weak evidence.

t.test(ctrl, trtm)

        Welch Two Sample t-test

data:  ctrl and trtm
t = 2.0769, df = 17.414, p-value = 0.05291
alternative hypothesis: 
 true difference in means is not equal to 0
95 percent confidence interval:
 -0.01260024  1.81260024
sample estimates:
mean of x mean of y 
      6.9       6.0 

Your background information says, "If your drug is effective, you will invest in its production, otherwise you will look for another drug. Assume that you obtained the following data (disease duration in days)." So it seems this is a preliminary feasibility study to see if the drug may have promise.

As you say, if you do a one-sided, pooled t-test, you can get a P-value below 5%.

t.test(ctrl, trtm, var.eq=T, alt = "g")

        Two Sample t-test

data:  ctrl and trtm
t = 2.0769, df = 18, p-value = 0.0262
alternative hypothesis: 
 true difference in means is greater than 0
95 percent confidence interval:
 0.1485724       Inf
sample estimates:
mean of x mean of y 
      6.9       6.0 

Just from the information in your Question, I am skeptical both about doing a one-sided test and about pooling. So this 'significance' barely at the 5% level seems to me more like P-hacking than like good statistical practice.


Finally, I will mention that if one were planning in advance to have an 90% or 90% probability of detecting an effect of about one day with standard deviations about one day, a 'power and sample size' procedure (this one from Minitab) shows that one should use about 20 subjects in each group instead of about 10. So the lack of clear cut significance should hardly have been a surprise. (Of course, power and sample size information depends on knowing effect size and standard deviations in advance. But in my experience trials of such a small size as this one are quite rare.)

Power and Sample Size 

2-Sample t Test

Testing mean 1 = mean 2 (versus ≠)
Calculating power for mean 1 = mean 2 + difference
α = 0.05  Assumed standard deviation = 1

            Sample  Target
Difference    Size   Power  Actual Power
         1      17     0.8      0.807037
         1      23     0.9      0.912498

The sample size is for each group.

enter image description here

Addendum per Comments: Six tests with P-values. Best practice is to decide before you see the data which one test meets assumptions and can tell you what you want to know. [Running multiple tests and 'cherry picking' the one with the smallest P-value is not appropriate.]

t.test(ctrl, trtm)$p.val
[1] 0.05290571   # Welch two-sided (AS ABOVE)
t.test(ctrl, trtm, alt="gr")$p.val
[1] 0.02645285   # Welch one-sided (P-val half) 
t.test(ctrl, trtm, alt="less")$p.val
[1] 0.9735471    # Welch one-sided (wrong side)

t.test(ctrl, trtm, var.eq=T)$p.val
[1] 0.05240201   # Pooled two-sided (slightly smaller than W)
t.test(ctrl, trtm, var.eq=T, alt="gr")$p.val
[1] 0.026201     # Pooled one-sided (P-val half, AS ABOVE)
t.test(ctrl, trtm, var.eq=T, alt="less")$p.val
[1] 0.973799     # Pooled one-sided (wrong side)
$\endgroup$
8
  • 1
    $\begingroup$ Thanks for the detailed explanation. For equal and unequal variances had a doubt in which to use. As equal variances was giving more degrees of freedom for less sample size, I went with it. Had a query if we use a two sided alternative, then if the null hypothesis is rejected in favor of alternative hypothesis we would not be able to tell if the drug is more effective as there can be cases where control group recovers faster $\endgroup$
    – ten do
    Jun 6 at 4:04
  • 1
    $\begingroup$ That's one reason most software programs give a confidence interval for the difference along with the P-value. If you do a two sided test and the result is significant, then you can look at the confidence interval to see which direction the effect goes. $\endgroup$
    – BruceET
    Jun 6 at 4:22
  • 1
    $\begingroup$ No. P-value for a two-sided test is larger than the P-value for the appropriate one-sided test. $\endgroup$
    – BruceET
    Jun 6 at 5:03
  • 1
    $\begingroup$ See addendum to my Answer showing just P-values for six tests. Best practice is to decide in advance of seeing data which one test answers (along with its CIs) what you want to know. $\endgroup$
    – BruceET
    Jun 6 at 7:36
  • 1
    $\begingroup$ Thank you for the detailed explanation $\endgroup$
    – ten do
    Jun 6 at 7:45

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.