Tell me more ×
Cross Validated is a question and answer site for statisticians, data analysts, data miners and data visualization experts. It's 100% free, no registration required.
up vote 1 down vote favorite

I have three groups of users $-$ $G1$, $G2$ and $Control$. The users in each of these three groups are different but are carefully selected and have similar properties. I treat each group with a unique treatment. $G1$ is treated by $T1$, $G2$ is treated by $T2$ and the $Control$ group is not treated at all. For each of these groups, I measure a variable $X$ before the treatment and after the treatment. I have the following questions -

[1] I would like to measure if my treatment (both $T1$ and $T2$) made a difference. In this case, I need to compare the variable $X$ for users in groups $G1 + G2$ to $Control$. Since I do not have any knowledge of the underlying distribution of data, I plan to use a Wilcoxon Signed Rank Test. Is this correct? If not, which other test would be appropriate?

[2] I want to measure which of the two treatments ($T1$ or $T2$) performed better. How do I do this? Should I perform a statistical test between users in $G1$ and $G2$ OR perform a statistical test between $G1$ and $Control$ AND $G2$ and $Control$ and interpret from the corresponding result?

share|improve this question

2 Answers

up vote 1 down vote

For the first problem, you can start with an ANOVA/regression and check the distributional assumptions (which are not about the data but about error as estimated by the residuals). If they are violoated you can then do a nonparametric test or robust test.

For the second problem, since your goal is to compare T1 to T2, you should compare G1 to G2. You can try a t-test to start.

Or you can do both within the ANOVA/Regression model using different contrasts.

share|improve this answer
For 1, Why is ANOVA a better choice? Pardon me but I am a beginner to this kind of statistics. For 2, Is it appropriate to compare G1 and G2 with a statistical significance test? – Dexter Nov 15 '12 at 11:39
2  
If the assumptions of ANOVA are met, it is slightly more powerful; it is also more familiar to most scientific audiences, which doesn't make it statistically better but is nevertheless useful. Also, the parameter estimates are easy to interpret; the W statistic is not intuitively meaningful. – Peter Flom Nov 15 '12 at 12:02
The only issue is that the ANOVA requires data to be normally distributed. I can not afford to make that assumption. – Dexter Nov 15 '12 at 12:11
@PeterFlom: you note that the distributional assumptions are not about the error as estimated by the residuals. I am confused. I always assumed that we needed independence of observations, normality and equality of variances in each group - and the last two are properties of the residuals. What did I misunterstand? – Stephan Kolassa Nov 15 '12 at 13:35
@StephanKolassa That was a mistake, I have fixed it. The assumptions are not about the data they are about the error. – Peter Flom Nov 15 '12 at 14:05
show 2 more comments
up vote 0 down vote

I went for Fisher's Exact Test and Chi-Square Contingency Test to answer my question. It looked more elegant and intuitive [0,1].

[0] http://en.wikipedia.org/wiki/Fisher's_exact_test

[1] http://math.hws.edu/javamath/ryan/ChiSquare.html

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.