1
$\begingroup$

I have three groups of users $-$ $G1$, $G2$ and $Control$. The users in each of these three groups are different but are carefully selected and have similar properties. I treat each group with a unique treatment. $G1$ is treated by $T1$, $G2$ is treated by $T2$ and the $Control$ group is not treated at all. For each of these groups, I measure a variable $X$ before the treatment and after the treatment. I have the following questions -

[1] I would like to measure if my treatment (both $T1$ and $T2$) made a difference. In this case, I need to compare the variable $X$ for users in groups $G1 + G2$ to $Control$. Since I do not have any knowledge of the underlying distribution of data, I plan to use a Wilcoxon Signed Rank Test. Is this correct? If not, which other test would be appropriate?

[2] I want to measure which of the two treatments ($T1$ or $T2$) performed better. How do I do this? Should I perform a statistical test between users in $G1$ and $G2$ OR perform a statistical test between $G1$ and $Control$ AND $G2$ and $Control$ and interpret from the corresponding result?

$\endgroup$

2 Answers 2

1
$\begingroup$

For the first problem, you can start with an ANOVA/regression and check the distributional assumptions (which are not about the data but about error as estimated by the residuals). If they are violoated you can then do a nonparametric test or robust test.

For the second problem, since your goal is to compare T1 to T2, you should compare G1 to G2. You can try a t-test to start.

Or you can do both within the ANOVA/Regression model using different contrasts.

$\endgroup$
7
  • $\begingroup$ For 1, Why is ANOVA a better choice? Pardon me but I am a beginner to this kind of statistics. For 2, Is it appropriate to compare G1 and G2 with a statistical significance test? $\endgroup$
    – Dexter
    Nov 15, 2012 at 11:39
  • 2
    $\begingroup$ If the assumptions of ANOVA are met, it is slightly more powerful; it is also more familiar to most scientific audiences, which doesn't make it statistically better but is nevertheless useful. Also, the parameter estimates are easy to interpret; the W statistic is not intuitively meaningful. $\endgroup$
    – Peter Flom
    Nov 15, 2012 at 12:02
  • $\begingroup$ The only issue is that the ANOVA requires data to be normally distributed. I can not afford to make that assumption. $\endgroup$
    – Dexter
    Nov 15, 2012 at 12:11
  • $\begingroup$ @PeterFlom: you note that the distributional assumptions are not about the error as estimated by the residuals. I am confused. I always assumed that we needed independence of observations, normality and equality of variances in each group - and the last two are properties of the residuals. What did I misunterstand? $\endgroup$ Nov 15, 2012 at 13:35
  • $\begingroup$ @StephanKolassa That was a mistake, I have fixed it. The assumptions are not about the data they are about the error. $\endgroup$
    – Peter Flom
    Nov 15, 2012 at 14:05
0
$\begingroup$

I went for Fisher's Exact Test and Chi-Square Contingency Test to answer my question. It looked more elegant and intuitive [0,1].

[0] http://en.wikipedia.org/wiki/Fisher's_exact_test

[1] http://math.hws.edu/javamath/ryan/ChiSquare.html

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.