2
$\begingroup$

How do I compare d-prime scores from 2 different conditions for the same individual, using the d-primes and standard errors?

$\endgroup$

2 Answers 2

3
$\begingroup$

Presumably your d' scores came from multiple "trials" on which your individual made a 2-alternative forced-choice response. For a single individual's data, I'd analyze as:

fit = glm(
    data = my_data
    , formula = response ~ reality*condition
    , family = binomial
)
summary(fit)

where "response" is the response they made on each trial, "reality" is the response they should have made on each trial, and condition is of course your condition variable. The intercept in this model reflects an overall response bias (ex. responding "A" more often than "B", regardless of reality). Unless they were guessing, you should get a main effect of reality (i.e. they made response "A" more often when reality was A and response "B" more often when reality was B). A main effect of condition would reflect an effect of condition on response bias, which can be of interest on it's own, but what you're likely most interested in is the interaction between reality and condition, which indexes the effect of condition on the discriminability of the two realities.

$\endgroup$
7
  • $\begingroup$ It's Go-NoGo (or A-not A) data so very similar. I don't have a lot of experience with glm but your explanation is clear enough that I think I can implement it in R. Thanks. However, as I'm tied to d-prime for historical reasons, ideally I would present it as a straightforward test of whether the dprimes were statistically different in the two conditions. $\endgroup$
    – Tiffany
    Oct 6, 2011 at 0:27
  • 1
    $\begingroup$ If you replace "family = binomial" with "family = binomial(link='probit')", then the interaction between reality and condition will represent a difference in d'. That is, if you collapse the "reality" effect to a difference score, it will be in d' units, and thus the interaction between reality and condition is about whether there is an effect of condition on d'. I tend to prefer the "family = binomial" (which is shorthand for "family = binomial(link='logit')" ) because I'm more familiar with the log-odds scale of the logit. $\endgroup$ Oct 6, 2011 at 12:35
  • $\begingroup$ Thanks for making the probit link explicit. It's been awhile since I've even thought about what d-prime actually is. I coded the glm as suggested and will use that going forward. I do have a further conceptual question, though- is it correct that the test of 'reality' is exactly equivalent to a t-test directly on the d-primes? Where the t-statistic is the difference between the d-primes divided by the SE of the difference. I originally did not think a t-test would be appropriate, since a d-prime is not a mean. But then, a d-prime is basically a transformation of a mean, right? $\endgroup$
    – Tiffany
    Oct 7, 2011 at 17:03
  • $\begingroup$ If you re-read my answer, you'll see that it is not the main effect of reality that evaluates a change in discriminability between conditions, but the reality:condition interaction. I'm not sure about a t-test on directly computed d' values would work; how would you calculate the denominator (SE od the difference)? $\endgroup$ Oct 7, 2011 at 18:34
  • $\begingroup$ Oops. The was a mistake, I meant reality:condition. That's what I looked at yesterday when I implemented your code and checked to see that the significance results matched my intuitions based on the d-primes & their SEs for the 2 conditions. SE of the difference I thought should as in the regular t-statistic be a weighted combination of the two SEs- sqrt(var(d-prime1)/n1 + var(d-prime2)/n2). I can of course run this myself and see if I get the same result as the glm, but was curious what you thought. I appreciate all your help! $\endgroup$
    – Tiffany
    Oct 7, 2011 at 20:52
0
$\begingroup$

Of course, you could literally look at the difference between the conditions and comment on the difference, because in some sense your sample is your population when you want to generalise only to one individual. However, usually in this context, you want to say that the difference is over and above some nuisance factors; measurement error is the main nuisance factor, but practice effects is another possibility.

I'm not sure if there is anything specific in the d-prime literature, but it sounds like you would want to read about indexes of "reliable change".

Many of these indices try to calculate whether the change observed is sufficiently large to rule out alternative interpretations (e.g., measurement error, practice effects, etc.). For example, a simple formula involves dividing actual change by the standard error of change, and seeing whether that which is observed is greater than some threshold (e.g., 1.96 for .05). However, there are a few thorny conceptual and estimation issues that you may want to consider.

$\endgroup$
2
  • $\begingroup$ Dividing the difference in d-primes by either the standard error of change or some pooled standard error was my vague feeling of how to proceed. The problem (I think) is I don't know how I would know what the threshold should be for a desired probability level, because I don't understand what distribution it is that results (that is, don't I need to know the sampling distribution of d-prime calculated over 1 individual's many trials like this?) $\endgroup$
    – Tiffany
    Oct 6, 2011 at 0:33
  • $\begingroup$ @Tiffany Some tests have norms, and these can guide estimation of the standard error; or if the individual is drawn from a larger sample, then you could derive an estimate of the standard error from that larger sample. $\endgroup$ Oct 6, 2011 at 0:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.