# Testing paired frequencies for independence

I hope this isn't either far too basic or redundant. I have been looking around for guidance but so far I am still uncertain of how to proceed.

My data consists of counts of a particular structure used in conversations between pairs of interlocutors. The hypothesis I want to test is the following: more frequent use of this structure by one speaker will tend to increase the frequency of the structure by the other speaker (i.e., this might be evidence of a priming effect).

So I just have two vectors, the counts for speaker A and the counts for speaker B are the columns, and if they are lined up each row represents a particular conversation, like this:

A B
0 1
0 2
1 0
3 1
0 2
2 0
2 1

There are about 420 conversations (rows). There are lots of zeros in this data.

What would be the best way to analyze this data? I am using R, if that makes a difference.

Here is a plot of the frequencies (counts). The x-axis is number of uses by speaker A, the y-axis number of uses by speaker B. The distinction between speakers means only that speaker A spoke first, and there's no special reason why they did. Otherwise the distinction between speaker A and speaker B is basically meaningless:

Valid XHTML http://phonematic.com/convplot.jpg

And this is frequency relative to number of sentences spoken by each speaker in each conversation. :

Valid XHTML http://phonematic.com/rs_plot.jpg

(I should mention that I have thrown out conversations with no hits at all, i.e. {0,0}.)

• Is each row the count of a different structure for the same conversation, or the count of the same structure for different periods of time? – RockScience Dec 6 '10 at 5:07
• Each row is the total number of usages of the same structure by two different people talking to each other. So, for instance, if I were counting passive sentences, then the number in column A would be the number of passive sentences used by speaker A, and the number in column B the number of passive sentences used by speaker B. So each row is a different conversation. Each conversation has exactly two participants. – Alan H. Dec 7 '10 at 4:13
• I also have the same data calculated relative to number of sentences spoken by each speaker in each conversation, if that makes a difference. – Alan H. Dec 7 '10 at 4:15
• Sorry if I'm a bit slow, but I still have some difficulty to understand your data structure. Are there only 2 speakers, with 420 repeated measurements collected on each (I mean one row=one type of conversation, but the same outcome is recorded, e.g. No. passive sentences)? I ask this because you have some kind of matching or pairing (between subjects A and B), but in this latter case, you would also have to deal with repeated measurements, and this renders marginal models for matched pairs less relevant. – chl Dec 7 '10 at 20:45
• No, these are all different speakers as well. This data was taken from a corpus of recorded telephone conversations. So speaker A in conversation 1 is not the same person as speaker A in conversation 2. – Alan H. Dec 7 '10 at 22:53

Log-linear models might be another option to look at, if you want to study your two-way data structure.

If you assume that the two samples are matched (i.e., there is some kind of dependency between the two series of locutions) and you take into consideration that data are actually counts that can be considered as scores or ordered responses (as suggested by @caracal), then you can also look at marginal models for matched-pairs, which usually involve the analysis of a square contingency table. It may not be necessarily the case that you end up with such a square Table, but we can also decide of an upper-bound for the number of, e.g. passive sentences. Anyway, models for matched pairs are well explained in Chapter 10 of Agresti, Categorical Data Analysis; relevant models for ordinal categories in square tables are testing for quasi-symmetry (the difference in the effect of a category from one case to the other follows a linear trend in the category scores), conditional symmetry ($\pi_{ab}<\pi_{ab}$ or $\pi_{ab}>\pi_{ab}$, $\forall a,b$), and quasi-uniform association (linear-by-linear association off the main diagonal, which in the case of equal-interval scores means an uniform local association). Ordinal quasi-symmetry (OQS) is a special case of linear logit model, and it can be compared to a simpler model where only marginal homogeneity holds with an LR test, because ordinal quasi-symmetry + marginal homogeneity $=$ symmetry.

Following Agresti's notation (p. 429), we consider $u_1\leq\dots\leq u_I$ ordered scores for variable $X$ (in rows) and variable $Y$ (in columns); $a$ or $b$ denotes any row or column. The OQS model reads as the following log-linear model:

$$\log\mu_{ab}=\lambda+\lambda_a+\lambda_b+\beta u_b +\lambda_{ab}$$

where $\lambda_{ab}=\lambda_{ba}$ for all $a<b$. Compared to the usual QS model for nominal data which is $\log\mu_{ab}=\lambda+\lambda_a^X+\lambda_b^Y+\lambda_{ab}$, where $\lambda_{ab}=0$ would mean independence between the two variables, in the OQS model we impose $\lambda_b^Y-\lambda_b^X=\beta u_b$ (hence introducing the idea of a linear trend). The equivalent logit representation is $\log(\pi_{ab}/\pi_{ba})=\beta(u_b-u_a)$, for $a\leq b$.

If $\beta=0$, then we have symmetry as a special case of this model. If $\beta\neq 0$, then we have stochastically ordered margins, that is $\beta>0$ means that column mean is higher compared to row mean (and the greater $|\beta|$, the greater the differences between the two joint probabilities distributions $\pi_{ab}$ and $\pi_{ba}$ are, which will be reflected in the differences between row and column marginal distributions). A test of $\beta=0$ corresponds to a test of marginal homogeneity. The interpretation of the estimated $\beta$ is straightforward: the estimated probability that score on variable $X$ is $x$ units more positive than the score on $Y$ is $\exp(\hat\beta x)$ times the reverse probability. In your particular case, it means that $\hat\beta$ might allow to quantify the influence that one particular speaker exerts on the other.

Of note, all R code was made available by Laura Thompson in her S Manual to Accompany Agresti's Categorical Data Analysis.

Hereafter, I provide some example R code so that you can play with it on your own data. So, let's try to generate some data first:

set.seed(56)
d <- as.data.frame(replicate(2, rpois(420, 1.5)))
colnames(d) <- paste("S", 1:2, sep="")
d.tab <- table(d$S1, d$S2, dnn=names(d)) # or xtabs(~S1+S2, d)
library(vcdExtra)
structable(~S1+S2, data=d)
# library(ggplot2)
# ggfluctuation(d.tab, type="color") + labs(x="S1", y="S2") + theme_bw()


Visually, the cross-classification looks like this:

   S2  0  1  2  3  4  5  6
S1
0     17 35 31  8  7  3  0
1     41 41 30 23  7  2  0
2     19 43 18 18  5  0  1
3     11 21  9 15  2  1  0
4      0  3  4  1  0  0  0
5      1  0  0  2  0  0  0
6      0  0  0  1  0  0  0


Now, we can fit the OQS model. Unlike Laura Thompson which used the base glm() function and a custom design matrix for symmetry, we can rely on the gnm package; we need, however, to add a vector for numerical scores to estimate $\beta$ in the above model.

library(gnm)
d.long <- data.frame(counts=c(d.tab), S1=gl(7,1,7*7,labels=0:6),
S2=gl(7,7,7*7,labels=0:6))
d.long$scores <- rep(0:6, each=7) summary(mod.oqs <- gnm(counts~scores+Symm(S1,S2), data=d.long, family=poisson)) anova(mod.oqs)  Here, we have$\hat\beta=0.123$, and thus the probability that Speaker B scores 4 when Speaker A scores 3 is$\exp(0.123)=1.13$times the probability that Speaker B have a score of 3 while Speaker A have a score of 4. I recently came across the catspec R package which seems to offer similar facilities, but I didn't try it. There was a good tutorial at UseR! 2009 about all this stuff: Introduction to Generalized Nonlinear Models in R, but see also the accompagnying vignette, Generalized nonlinear models in R: An overview of the gnm package. If you want to grasp the idea with real data, there are a lot of examples with real data sets in the vcdExtra package from Michael Friendly. About the OQS model, Agresti used data on Premarital Sex and Extramarital sex (Table 10.5, p. 421). Results are discussed in §10.4.7 (p. 430), and$\hat\beta$was estimated at -2.86. The code below allow (partly grabbed from Thompson's textbook) to reproduce these results. We would need to relevel factor levels so as to set the same baseline than Agresti. table.10.5 <- data.frame(expand.grid(PreSex=factor(1:4), ExSex=factor(1:4)), counts=c(144,33,84,126,2,4,14,29,0,2,6,25,0,0,1,5)) table.10.5$scores <- rep(1:4,each=4)
summary(mod.oqs <- gnm(counts~scores+Symm(PreSex,ExSex), data=table.10.5,
family=poisson)) # beta = -2.857
anova(mod.oqs) # G^2(5)=2.10

• Wow, this looks really useful. I will acquire a copy of this book as soon as I can, since everyone seems to be citing it. In the meantime, just a naive question: can these models deal with an arbitrary number of random effects? I think I need 3 in my model. – Alan H. Dec 10 '10 at 18:03
• @Alan Be aware that we (@caracal and me) are not citing the same book. CDA is the more complete one, but Laura Thompson's textbook already includes about 10 pages of summary for each chapter. Look at her textbook first. ICDA is available on Google books I think. – chl Dec 10 '10 at 20:38
• Typo in conditional symmetry definition? $\pi_{ab}<\pi_{ba}$ or $\pi_{ab}>\pi_{ba}$, $\forall a<b$ – Scortchi - Reinstate Monica Aug 2 '17 at 9:15

You seem to have ordered categorical data, therefore I suggest a linear-by-linear test as described by Agresti (2007, p229 ff). Function lbl_test() of package coin implements it in R.

Agresti, A. (2007). Introduction to Categorical Data Analysis. 2nd Ed. Hoboken, New Jersey: John Wiley & Sons. Hoboken, NJ: Wiley.

cor(x=A, y=B, method = "spearman")

• Both assume continuous variables, so that the probability of ties within a sample is $0$. With the given data, there will be a lot of ties. – caracal Dec 6 '10 at 10:37