# Testing the significance of differences in frequencies across treatments using $\chi^2$ test

I've run a small experiment and need to test the data by comparing the frequencies of different treatments, specifically, whether they are significantly different. I think the chi-squared test is appropriate, but I am not sure how to execute it because it's somewhat outside the range of my course.

Simplifying, the data looks something like this:

               A Choices (%)       B Choices (%)       Observations
Treatment 1:       40                   60                  n=30

Treatment 2:       65                   35                  n=30

Treatment 3:       70                   30                  n=30


How exactly would I conduct a chi-sq test?

Is it the best test for investigating differences in the frequencies of 'A' between treatment pairs?

Say, testing H0: Treatment1(% A Choices) = Treatment3 (% A Choices)

Is it also possible to do a test like H0: Treatment2(% A Choices) = Treatment3(% A Choices) > Treatment1(% A Choices)

Thanks!

-
Hi Verity and welcome to the site. Could you provide some more information about your study, for example are the same people in Group A (and Group B) undergoing all three treatments? Could you also explain what the "A Choices" and "B Choices" mean? – Michelle Mar 17 '12 at 2:34
Different people undergo each treatment. The study is a multistage game of chance, culminating in participants choosing between Option A - receiving a certain amount of money, or Option B - taking a risky gamble with the chance to win a larger amount of money. The different treatments vary aspects of the game, and are expected to result in different distributions of A and B choices. I need to compare Treatments 1 and 2 to Treatment 3, separately, in order to see if they are significantly different. So A or B refers to their choices ie. the data outcome. – Verity Mar 17 '12 at 9:01

@Michelle raises a good question--it depends on what the answer is. However, at first blush, I suspect that you could just multiply your percentages by your N's and get counts for each cell. Then you would have a straightforward 3x2 chi-squared test setup. I get the following:

                A      B
Treatment 1     12     18
Treatment 2     11?    19?
Treatment 3     21      9


(Note that 65% (& 35%) times 30 does not yield a whole number, so I suspect there's a little more to it than exactly this.) The basic chi-squared test of independence checks to see if knowing what row a unit belongs to helps you know what column it belongs to (and vice versa). The wikipaedia page contains all the info you need on the test and how to conduct it.

It is possible to test just 1 vs. 3, and just 2 vs. 3; however, these tests wouldn't be independent. I should think a simple chi-squared test is sufficient.

-
Thanks, I'll have another look at that. Do you know any textbooks that are more detailed, as that would be really useful? – Verity Mar 17 '12 at 9:06
Any intro to stats textbook will cover how to run a basic chi-squared test. If you want a more specialized treatment, the first 2 chapters of Agresti's Introduction to Categorical Data Analysis will do the trick. – gung Mar 17 '12 at 13:45
Online you can find an extremely detailed description of conducting a chi-squared test at psychstat.missouristate.edu/introbook/sbk28m.htm. – whuber Mar 17 '12 at 14:40