Proper test for between-group percentage difference and for individual response differences? I have two groups (treatment and control) and intend to measure how a treatment affects people's choices. The choice is simply 'yes' or 'no' so t-test does not really seem fit here. Hence, I would like to see if the percentage of choosing 'yes' is significantly different in the treatment group from that percentage in the control group. The design is between-subjects and each group should have around 500 responses. What test should be the most proper here? Maybe Chi-squared?
Another question I have is for another survey where people rating items in a 7-degree scale. Is there any test to tell whether, for an individual, his/her rating on one item is significantly higher/lower than on the rest items?
Thanks!
 A: For the first, a test for a difference in two binomial proportions for $H_0: p_t = p_c$ against $H_a: p_t \ne p_c$ would look like this in R:
Suppose the treatment group has 325 Yes responses out of 501 and the
control group has 278 Yes responses out of 489. The treatment group
has about 65% Yes's and the control group only about 57% Yes's. So
the proportion of Yes's in the treatment group is somewhat larger.
The question is whether the difference is large enough to be
statistically significant.
prop.test(c(325, 278), c(501, 489), cor=F)

    2-sample test for equality of proportions 
    without continuity correction

data:  c(325, 278) out of c(501, 489)
X-squared = 6.6843, df = 1, p-value = 0.009727
alternative hypothesis: two.sided
95 percent confidence interval:
 0.0195785 0.1408124
sample estimates:
   prop 1    prop 2 
0.6487026 0.5685072 

The P-value about 0.01 indicates that significance
at the 1% level (thus surely at the 5% level). That is, $H_0$ is rejected.

I'm not sure I understand what data you have for the second situation.
If you have 20 questions on a 7-point Likert and John has 19 responses
5 and below and one 7, it is obvious that the 7 is higher than the rest,
but I'm not sure what statistical significance means in this situation.
Do several other respondents have similar 'unusual' scores? If so, is the same question typically rated higher?
It's unclear how much higher
catches your attention; whether we are talking about John alone
or in comparison with other subjects; whether this particular
question was of interest from the start or we noticed it after the survey.
Maybe descriptive methods are best here. Consider the boxplot for
John's (hypothetical) questionnaire:
j = c(1,3,2,4, 4,3,2.2, 7,4,5,1, 4,4,3,3, 3,5,4,2)
boxplot(j, horizontal=T)


