Hypothesis testing for count variables in (possibly) nested data

Question

I am analyzing word usage in multiple two-person conversations between students and teachers, and want to test whether the usage of a given word differs between the two speaker roles. My interactions have been grouped together to form a single (27,000x2) dataframe. Each row in the dataframe reflects a sentence spoken by either a teacher (T) or a student (S). There are two columns, one containing the label of the speaker (S or T), and the other the count of a word X in that sentence.

As you can imagine, my data contains a lot of zeros...

------------------
 speaker|count(x)
------------------
    S       0
    T       1
    S       0
    T       3
    S       2
    T       0
    S       0
    T       1
------------------

Put simply, I want to know if teachers usage of word X differs from students. I am also concerned that as my data has grouped multiple conversations into a single dataframe, I might have to account for the nested structure of the data (although I could be wrong).

I am assuming that a simple t-test is not the most appropriate way to deal with this - working with counts = outliers, normality violations etc. - but I'm unsure where to turn next.

Answer 1

Firstly, this is not a good way to represent your data, since it entails a serious loss of information. From the way you have represented your data, it is not possible to identify the pairings of the counts in each conversation. Since the conversations here each have two parties, your data should pair the student with the speaker and show the count for each party in each conversation. So it should look something like this instead:

--------------------------------
conversation | count.T | count.S
           1         1         0   
           2         3         0
           3         0         2
         ...
--------------------------------

Since you have a lot of data, and the observed counts are generally low, the simplest thing here would be to form a contingency table showing the number of outcomes with each pair of counts. You can plot this data as a bubble plot showing the proportion of outcomes in each pair, and this will give you a graphical depiction of the joint distribution of the two count values. That will be much more informative than just doing a statistical test comparing the marginal distributions.

Now, if you particularly want to compare the marginal distributions to see if they are the same then you can do this using a chi-squared test using a simulated computation of the p-value for the test. You can do this in R using the chisq.test function by setting simulate.p.value = TRUE.