1
$\begingroup$

I accomplished a field experiment which results in six differently treated groups (including one control group, each n = 300). Due to randomization all groups are expected to be equal in terms of their characteristics (average price, average customer age, etc.). The only difference is supposed to be the treatment each group received. The resulting dependent variable (outcome) is binary (1/0).

How can I now compare the six groups in terms of their binary outcome? I want to know the respective probability of a treatment leading to outcome 1.

Is a Kruskal-Wallis test adequate or should I rather use logistic regression? Using logistic regression also involves control variables although these should be distributed equally among all groups?

Thank you very much!

Improve this question
$\endgroup$

2 Answers 2

Reset to default
1
$\begingroup$

The way to do this is with logistic regression with the group being the independent variable. You probably will want to control for other variables too.

Improve this answer
$\endgroup$
2
  • 1
    $\begingroup$ But everything accept of the treatment is equally distributed among all groups, isn't it? So the only difference is the treatment within each group. So why should there be a need to control for other variables than the group itself? And wouldn't it be easier to just check if the mean (which represents the probability to end in ouctome 1) is statistically different from the respective other means? This way it would be easily possible to quantify the percentual difference among the groups/treatments in terms of their probability to end up in outcome 1. Right? $\endgroup$
    – Lukas
    Apr 13, 2016 at 11:35
  • 1
    $\begingroup$ Because the control variables might affect the parameter estimate for the group. There could also be interactions. $\endgroup$
    – Peter Flom
    Apr 14, 2016 at 10:51
1
$\begingroup$

You can organize the data as a 2 by 6 contingency table. Descriptively you can present this table (absolute counts) and the proportion of "1" (relative counts) per group. Different proportions highlight treatment differences. If it makes sense to test the null hypothesis of equal group proportions, you can e.g. use a classic chi-squared test of association (which is similar to the likelihood ratio test of the logistic regression). Maybe you rather want to compare each of the five test groups against the control. Then you can go for five tests. Another strategy is to test each treatment against each other (again e.g. with chi-squared test). Anyway you will need a prespecified strategy to correct for multiple testing (or not), like Bonferroni-Holm. It really depends on the research question.

Some example R code

# input: prepare data and check contingency table
set.seed(39)
n <- 300*6
group <- rep(LETTERS[1:6], n/6)
outcome <- sample(0:1, n, T)

(tab <- table(outcome, group))

# output
       group
outcome   A   B   C   D   E   F
      0 158 156 144 146 160 151
      1 142 144 156 154 140 149

# input, cont.: Proportions of "1" across groups
round(prop.table(tab, 2), 3)

# output
       group
outcome     A     B     C     D     E     F
      0 0.527 0.520 0.480 0.487 0.533 0.503
      1 0.473 0.480 0.520 0.513 0.467 0.497

# input: test for association
chisq.test(tab)

# output
X-squared = 2.8741, df = 5, p-value = 0.7194

In this data set, the proportions of "1" do not differ much across groups (always between 47% and 52%). No true group differences are detected at the 5% level.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks a lot for your answer. I am not familiar with R as I use STATA, but your output (groups A - E) seems to go into the right direction. My goal is to lets say find out which group tends to become more easily sick (coded 1). I'd like to say, Group A has a 3% higher chance to get sick than group E. I therefore do not only want to compare the groups with the control group, but also with each other. Does that make sense? Do you recommend a chi sqd test or logistic Regression? I wanto to go beyond descriptive statistics and be able to draw conlusions for the whole population. $\endgroup$
    – Lukas
    Apr 13, 2016 at 14:14
  • $\begingroup$ chi-squared test and the test from logistic regression agree, so which one to choose is irrelevant. If you want to compare each group with each other, then you need to have a clear strategy regarding multiple testing. Of course you can do a chi-squared test for each pair of treatments and then do some sort of Bonferroni-Holm correction to identify "different" pairs. $\endgroup$
    – Michael M
    Apr 13, 2016 at 14:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.