3
$\begingroup$

I have data from an experiment that consist of nominal-scale responses that subjects made in two treatments (the treatments are between subjects, not repeated measures). I've provided my data in data frame and table form below. My goal is to compare the distributions of the responses in the two treatments - t1 and t2 - to see if they are significantly different from one another. I am not interested in differences among levels of the response variable (e.g., between response a and c).

I'm unsure how to approach this. My first thought was a Fisher exact test of the 8x2 contingency table (tab below), which yields: p-value = 0.2102. However, it seems that this tests whether the response variable is associated with the treatment variable as a whole, rather than testing for a difference between the treatments. Any suggestions for a better approach are most welcome.

# data frame with treatments and responses
df <- structure(list(id = 1:30, treatment = structure(c(2L, 1L, 2L, 
    1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 
    1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L, 2L, 1L), .Label = c("t1", 
    "t2"), class = "factor"), response = structure(c(2L, 1L, 1L, 
    1L, 1L, 1L, 6L, 8L, 1L, 1L, 4L, 1L, 1L, 1L, 5L, 1L, 7L, 1L, 8L, 
    1L, 4L, 1L, 4L, 2L, 1L, 1L, 1L, 3L, 1L, 1L), .Label = c("a", 
    "b", "c", "d", "e", "f", "g", "h"), class = "factor")), .Names = c("id", 
    "treatment", "response"), row.names = c(1L, 2L, 3L, 4L, 5L, 6L, 
    7L, 8L, 9L, 10L, 11L, 12L, 13L, 14L, 15L, 16L, 17L, 18L, 19L, 
    20L, 21L, 22L, 23L, 24L, 25L, 26L, 27L, 28L, 29L, 30L), class = "data.frame")

# 8x2 contingency table
tab <- structure(c(12L, 1L, 1L, 0L, 0L, 0L, 0L, 1L, 7L, 1L, 0L, 3L, 
    1L, 1L, 1L, 1L), .Dim = c(8L, 2L), .Dimnames = structure(list(
    response = c("a", "b", "c", "d", "e", "f", "g", "h"), treatment = c("t1", 
    "t2")), .Names = c("response", "treatment")), class = "table")
Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

However, it seems that this tests whether the response variable is associated with the treatment variable as a whole, rather than testing for a difference between the treatments.

Those are the same thing. Each implies the other.

Improve this answer
$\endgroup$
6
  • $\begingroup$ I kind of see what you mean after thinking about it more. Is there a way to get an idea of the magnitude of the difference between treatments, as opposed to just whether the difference is significant? An effect size? $\endgroup$
    – Chris
    Commented Feb 11, 2013 at 3:53
  • $\begingroup$ There are many different ways the distribution of treatments can differ! What sort of effect do you want to measure? $\endgroup$
    – Glen_b
    Commented Feb 11, 2013 at 3:55
  • $\begingroup$ The key thing is where the modal values lie. In the above example, the modes for treatment t1 and t2 are both at response a. I'm interested in how different the distributions are if say the mode of t1 was b and t2 was g. $\endgroup$
    – Chris
    Commented Feb 11, 2013 at 4:02
  • 1
    $\begingroup$ I agree. Perhaps I'll calculate the odds for each response in t1 vs t2. The change in the distribution of the odds over the responses may give some insight. Or perhaps even the difference between the odds for t1 and t2. $\endgroup$
    – Chris
    Commented Feb 11, 2013 at 4:20
  • 2
    $\begingroup$ I often find that simple graphics convey much more information that a (necessarily low-dimensional) measure of effect size. For instance, I find plot(tab) already quite enlightening, although something else may, of course, be more appropriate to your research question. $\endgroup$
    – Stephan Kolassa
    Commented Feb 11, 2013 at 8:53

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.