1
$\begingroup$

I already came across this question: Difference in means vs difference in proportions, but could not answer my question based on the answers and comments there.

My question would be this: Suppose there are two groups, A and B, whose participants are asked to write texts. Let's say the participants of group A have produced a total of $90,000$ of characters and the participants of group B have produced $100,000$. I also know the number of characters per participant. Now I want to compare the results regarding the number of characters between the groups. Two things are not clear to me:

(1) What would, in this case, a binomial test check with an assumed probability of $0.5$ (and a present probability of $90,000 / 190,000 = 0.47$)?

(2) In comparison, what would a $t$-test check based on the individual character counts of the participants?

Possibly, Fisher's exact test or a $\chi^2$-squared test could take the place of the binomial test. Still, I wonder from what different "perspectives" the data are being viewed here, when either the total counts or the mean counts per participant are compared.

Improve this question
$\endgroup$
6
  • $\begingroup$ what do you mean by frequency here? And what's your end goal? to test whether there is a significant difference between the two groups in terms of characters generated per person? $\endgroup$
    – Amin Shn
    Commented Jun 19, 2022 at 12:18
  • $\begingroup$ Thank you for checking back and sorry for the imprecise wording. By frequency, I mean counts. My end goal would be to test if there is a significant difference between the two groups in terms of characters generated in general, so to speak. And I wonder if both tests would be useful, and if so, how they would have to be interpreted. I realize that the binomial test would look more at the ratio of total counts and the t-test would look at means and variances, but I'm not clear under what circumstances one test would be preferable to the other. $\endgroup$
    – alex
    Commented Jun 19, 2022 at 12:38
  • $\begingroup$ I should perhaps mention that I am conducting an exploratory data analysis, not a hypothesis testing study. $\endgroup$
    – alex
    Commented Jun 19, 2022 at 12:47
  • $\begingroup$ If you find a significant difference between the two groups in terms of characters generated per person that directly translates into a significant difference between the groups in terms of characters generated in general when the groups have the same sample size. Binomial test does not make sense to me at all in your problem (unless I misunderstood your question). $\endgroup$
    – Amin Shn
    Commented Jun 20, 2022 at 11:22
  • $\begingroup$ Thank you so much for highlighting that a binomial test is not useful at this point! Which I now fully realize after some thought. I was completely on the wrong track. Thanks also for clarifying that differences in characters generated per person also mean group differences, given equal group sizes. $\endgroup$
    – alex
    Commented Jun 22, 2022 at 9:54

1 Answer 1

Reset to default
0
$\begingroup$

If I understand correctly, the "binomial test" is simply examining whether both groups have the same probability of producing a character. The way you stated the test, there is no accounting for differences among individuals within each group. You could have 1 individual in each group or 10,000 in each group and would get the same result.

The t-test takes the variance among individuals within a group into account. That should be more generally useful than just analyzing the overall number of characters per group.

Improve this answer
$\endgroup$
1
  • $\begingroup$ Thank you very much for your reply and highlighting that a $t$-test is generally more useful here! Through your explanations and also through your highlighting ("binomial test", groups) as well as through @Amin's comments I now understand that actually only a $t$-test, in any case not a binomial test, comes into question in this case. $\endgroup$
    – alex
    Commented Jun 22, 2022 at 10:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.