# Comparing two logit or probit curves using a single parameter

I've conducted a psychological experiment on the same subject, under two different condition. For each condition I've collected the number of correct and wrong answer for each stimulus (number of trials per stimulus = 10; number of stimulus = 15). From the data collected I fitted a psychometric curve (probit or logit curve), and now I want to compare the results.

Consider subject 1: His answers are collected in the following matrices

**COND_1**
correct
cnt   Y  N
1   0 10
2   0 10
3   0 10
4   0 10
5   0 10
6   0 10
7   1  9
8   4  6
9   5  5
10  7  3
11  8  2
12  7  3
13 10  0
14 10  0
15 10  0

**COND_2**
correct
cnt   Y  N
1   0 10
2   2  8
3   2  8
4   3  7
5   2  8
6   4  6
7   8  2
8  10  0
9   8  2
10 10  0
11  9  1
12 10  0
13 10  0
14 10  0
15 10  0

COND_1.1 <- matrix(c(0, 2, 2, 3, 2, 4, 8, 10, 8, 10, 9, 10, 10, 10, 10, 10, 8, 8, 7, 8, 6, 2, 0, 2, 0, 1, 0, 0, 0, 0), byrow=F, ncol=2)
COND_1.2 <- matrix(c(0, 0, 0, 0, 0, 0, 1, 4, 5, 7, 8, 7, 10, 10, 10, 10, 10, 10, 10, 10, 10, 9, 6, 5, 3, 2, 3, 0, 0, 0), byrow=F, ncol=2)

cnt <- seq(from=0, to=1.4, by=0.1)

ddprob.1.1 <- glm(COND_1.1 ~ cnt, family = binomial(link = "probit"))
ddprob.2.1 <- glm(COND_2.1 ~ cnt, family = binomial(link = "probit"))


Observing the plot, you can surely say that performance under condition 2 is better then the performance under condition 1. In fact if you compute some parameters like AUC, p25, p50, p75 (the value of the stimulus at which the number of correct answers are 25%, 50% and 75% each) [threshold parameters], you'll see that:

COND_1      COND_2
AUC    <    AUC
p25    >    p25
p50    >    p50
p75    >    p75


Now consider subject 2:

COND_1
correct
cnt   Y  N
1   0 10
2   0 10
3   0 10
4   0 10
5   3  7
6   3  7
7   4  6
8   5  5
9   9  1
10  9  1
11 10  0
12 10  0
13 10  0
14 10  0
15 10  0

COND_2
correct
cnt   Y  N
1   0 10
2   1  9
3   0 10
4   4  6
5   2  8
6   6  4
7   4  6
8   7  3
9   4  6
10  7  3
11  7  3
12 10  0
13  9  1
14 10  0
15 10  0

COND_1.2 <- matrix(c(0, 0, 0, 0, 3, 3, 4, 5, 9, 9, 10, 10, 10, 10, 10, 10, 10, 10, 10, 7, 7, 6, 5, 1, 1, 0, 0, 0, 0, 0), byrow=F, ncol=2)
COND_2.2 <- matrix(c(0, 1, 0, 4, 2, 6, 4, 7, 4, 7, 7, 10, 9, 10, 10, 10, 9, 10, 6, 8, 4, 6, 3, 6, 3, 3, 0, 1, 0, 0), byrow=F, ncol=2)
cnt <- seq(from=0, to=1.4, by=0.1)

ddprob.1.1 <- glm(COND_1.2 ~ cnt, family = binomial(link = "probit"))
ddprob.2.1 <- glm(COND_2.2 ~ cnt, family = binomial(link = "probit"))


The plotted curves are shown in the image below.

Can you say what performance is better? Under condition 1 or under condition 2? Extrapolating AUC, p25, p50, p75, you have:

COND_1      COND_2
AUC     >   AUC
p25     >   p25
p50     =   p50
p75     <   p75


So my question is: Is there a method to say that a generic curve (logit, probit or whatelse) is HIGHER then another? Is there a method to compare (a single number of "performance") that describe the differences of the two curves? My example shows that AUC, p25, p50, p75 are not good parameters. I'd like to compute a single numeric parameter for each curves, to make a simple t-paired test, extrapolated from the distribution seen.

• If in fact subject #2 performs better under Condition 1 for high values of the stimulus & better under Condition 2 for low values of the stimulus, do you really want a single number to sum up performance? – Scortchi - Reinstate Monica Apr 13 '13 at 16:23
• @Scortchi Maybe not (in my overall design, however, one value is sufficient), but if so, i could find a threshold of inversion of performances. But the new problem becomes: how to extract 2 parameters (one for low values of the stimulus, and one for high values) for describing performance with a statistical approach? (and not just "viewing" the plot) – Tommaso Apr 13 '13 at 16:29
• Well I'd be thinking about using logistic regression with performance as the response, & with subject, condition, stimulus, & their interactions as the predictors. – Scortchi - Reinstate Monica Apr 13 '13 at 16:32
• How can one parameter possibly be sufficient? Or even two? And it's debatable whether those curves are proper: What is the "stimulus" and how does each stimulus vary? Your graph assumes that stimulus is interval level data (that is, that the difference between e.g. stimulus 2 and 3 is the same as between stimulus 4 and 5). – Peter Flom - Reinstate Monica Apr 13 '13 at 19:22
• @PeterFlom I've added some infos in the comment to the Scortchi answer. Yes, the stimulus is continuous, and yes the interval level data is constant. In the first graph it appears to change the intercept, while in the second graph it appears to change the slope. How can the overall performance of SUBJ-n be described properly? – Tommaso Apr 15 '13 at 10:38

$$\log\frac{\pi}{1-\pi}=\beta_0 +\beta_p p + \beta_c c + \beta_s s + \beta_{pc} pc + \beta_{ps} ps + \beta_{cs} cs$$
where $\pi$ is the probability of success, $p$ is a dummy variable for person, $c$ is a dummy variable for stimulus, & $s$ is the strength of the stimulus. Each of the estimated coefficients (the $\beta$s) tells you something useful, & you can use a likelihood ratio test instead of a t-test.
• I don't know if the interaction is physiologically plausible. You can test the significance of the $cs$ term. It could suggest there's something wrong with the experimental protocol - sequence effects perhaps? – Scortchi - Reinstate Monica Apr 14 '13 at 11:05