1
$\begingroup$

I have data from a relatively easy task where subjects had to detect a signal in noise. I computed the hit and false alarm rates (HR, FAR), and then proceeded to compute d' using the standard formula, found e.g. in Green&Swets 1966, and described also here:

d' = z(FAR) – z(HR),

where the z-scores for the left-tail p-values from the normal distribution are computed in Matlab as:

z(HR)  = norminv(1-HR, 0, 1)
z(FAR) = norminv(1-FAR, 0, 1).

This produced all-negative values for d', and I know the usual practice is to then inverse the sign in the d' formula, to get positive d' values:

d' = z(HR) – z(FAR)

As a sanity check, I created a cross-subjects scatterplot of raw HR values against d', expecting to find a by-and-large positive correlation. However, what I found was a very neat negative correlation:

enter image description here

I know the p-value in the above formulas for the z-scores are taken by some not as 1-p but simply as p, thus I recalculated z(HR) and z(FAR) as

z(HR)  = norminv(HR, 0, 1)
z(FAR) = norminv(FAR, 0, 1)

To have positive d' values, I went back to its z(FAR)-z(HR) formula, but the values & their plot end up looking the same as the screenshot above.

Surely, assuming moderate FARs across the board, subjects with the highest HRs should in principle also have the highest d' values?! What am I getting confused here?

REFERENCE: Green, D. M., & Swets, J. A. (1966). Signal detection theory and psychophysics (Vol. 1). Wiley New York.


EDIT: Having done more sanity checks on the data, the calculation of the FARs was in fact erroneous, and this was leading to the absurd negative correlation described. Once that fixed, the formulas above (with p rather than 1-p for norminv, and polarity HR-FAR rather than FAR-HR) gives sensible d' values and a positive correlation. I will post this an answer as well.

$\endgroup$
  • $\begingroup$ norminv computes the normal inverse cumulative distribution function; it appears in the d' formula at de.mathworks.com/matlabcentral/fileexchange/… $\endgroup$ – z8080 Jul 10 '20 at 18:53
  • $\begingroup$ "other than just because it occurs somewhere else" - clearly, you have to start from some formula occurring somewhere, don't you. The d' formula is the same (and involves the normal inverse cumulative distribution function) not just at that link but in other places too, e.g. en.wikipedia.org/wiki/Sensitivity_index ; if you've no time to follow any links, not sure what else to say. Will try to obtain the sanity check plots you suggested. $\endgroup$ – z8080 Jul 10 '20 at 19:00
  • $\begingroup$ also, in SDT it's HR and MR that add up to 1, and not HR+FAR $\endgroup$ – z8080 Jul 10 '20 at 19:02
0
$\begingroup$

You could plot the ROC curves. On the plot with the zROC curve the $d^\prime$ relates to the distance between the curve and the straight line zTPR=zFPR.

If the variances of the signals and noise are equal, then the slope of the zROC curve is 1, and you get a constant $d^\prime$ as function of TPR. But you get a different slope when the variances are not equal.

Your negative correlation between TPR and $d^\prime$ means that the slope of the zROC curve is <1.

example

TPR: true positive rate or hit rate

FPR: false positive rate or false alarm rate

layout(matrix(1:9,3))

t = seq(-10,10,0.1)

FPR = rbinom(201,1000, p = 1 - pnorm(t,0,1))/1000
TPR = rbinom(201,1000, p = 1 - pnorm(t,2,2))/1000

plot( FPR,TPR )
lines(c(-10,10),c(-10,10), lty=2)

zFPR = qnorm(FPR)
zTPR = qnorm(TPR)

plot(zFPR, zTPR)
lines(c(-10,10),c(-10,10), lty=2)

dp = zTPR-zFPR
plot(dp,TPR)

Intuition: The z-scores of TPR and FPR are changing, when the triggerlevel of declaring a detected signal is changing. This change of the two z-scores can be at different rates, and that is the case when the variance of the signal and noise are different. If zFPR drops faster than zTPR, then the $d^\prime$ will increase for lower TPR.

$\endgroup$
  • $\begingroup$ Thanks. This was taking me in quite a different direction, but I did more checks on the data and found what my problem was (see edited question). $\endgroup$ – z8080 Jul 13 '20 at 6:57
0
$\begingroup$

My bad. After doing more sanity checks on the data, the calculation of the FARs was in fact erroneous, and this was leading to the absurd negative correlation described. Once that fixed, the formulas above (with p rather than 1-p for norminv, and polarity HR-FAR rather than FAR-HR) gives sensible d' values and a positive correlation.

$\endgroup$
  • $\begingroup$ But don't you have negative d' values now with HR-FAR? "I know the p-value in the above formulas for the z-scores are taken by some not as 1-p but simply as p, thus I recalculated z(HR) and z(FAR).... To have positive d' values, I went back to its z(FAR)-z(HR) formula, but the values & their plot end up looking the same as the screenshot above." $\endgroup$ – Sextus Empiricus Jul 13 '20 at 8:27
  • $\begingroup$ I believe that the negative correlation is not strange, and actually very normal. $\endgroup$ – Sextus Empiricus Jul 13 '20 at 8:29
  • $\begingroup$ no, I now have (mostly) positive d' values. What I said before was just an artefact of the erroneous FARs, which were all close to 1 due to a coding bug of mine. And I don't think it'snormal to have a neg corr betwenn d' and HR, as high HR should generally mean high d', as I was saying. $\endgroup$ – z8080 Jul 13 '20 at 12:39
  • $\begingroup$ z8080, the high HR also means high FAR, it is not necessarily that you get high d' with high HR. Actually, generally you get low d' with high HR. I believe that the second graph in my answer, the zROC curve with a slope less than 1, is very typical. It means that the variance with the signal/stimulus is larger than the variance without the signal/stimulus (which makes sense when the stimulus is not always the same, and has variance in the stimulus plus the noise which is always present). $\endgroup$ – Sextus Empiricus Jul 13 '20 at 16:12
  • $\begingroup$ "it is not necessarily that you get high d' with high HR" - that's true, as the FAR comes into play. However, I think (though I cannot prove it off the top of my head) that more often than not, d' will correlate positively with HR, not negatively. I believe you are also wrong with "the high HR also means high FAR". In an earlier comment (that you later deleted), you stated that HR and FAR should add up to 1 (which is clearly not the case, as I pointed out), thus please allow me to maintain some degree of doubt. $\endgroup$ – z8080 Jul 13 '20 at 17:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.