3
$\begingroup$

For each subject in my sample, I need to compute a sensitivity index, d-prime, defined in Signal Detection Theory as

d' = z(HR) - z(FAR)

where HR and FAR are the hit- and false-alarm-rates respectively. I am confused about why and how each of these rates need to be standardised.

These rates are each expressed as a percentage, i.e. they are scalars, whereas z-scores are, as I understand, computed for a vector of scalars (a distribution).

It is also unclear to me whether the standardisation is to be made with respect to the other scores in the sample (other subjects), or just with respect to a standard normal distribution, N(0,1).

_____LATER EDIT:

What is in fact the cost (the mistake) in defining a measure of performance just as the HR-FAR difference, i.e. with non-standardised rates? For example, a study by Patel et al. 2008 define in this manner performance in a music task where normal and anomalous chord sequences have to be discriminated:

Performance was measured as % hits - % false alarms. A hit was defined as an anomalous sequence classified as such, while a false alarm was a normal sequence classified as anomalous enter image description here

Patel et al. seem to use this measure as a sensitivity index (d'), although they do not actually call it as such.

It seems like in both cases (scores standardised or not), chance-level responding still leads to a score of 0 - is this is not a sanity check that both measures are equally valid?

References:

Patel, A. D., Iversen, J. R., Wassenaar, M., & Hagoort, P. (2008). Musical syntactic processing in agrammatic Broca’s aphasia. Aphasiology, 22(7–8), 776–789. https://doi.org/10.1080/02687030701803804

$\endgroup$
4
  • 1
    $\begingroup$ I think you are confusing two of the many things which $z$ stands for. What you want here is the normal deviate corresponding to $p$. $\endgroup$
    – mdewey
    Commented Nov 30, 2017 at 13:38
  • $\begingroup$ Sorry, I don't think I understand which p that is. Is each subject's score to be standardised (z-scored) with respect to the other scores in the sample, or just with respect to N(0,1)? $\endgroup$
    – z8080
    Commented Dec 3, 2017 at 11:46
  • 1
    $\begingroup$ The second one. $\endgroup$
    – mdewey
    Commented Dec 3, 2017 at 11:56
  • $\begingroup$ OK, I still don't understand why that is, but if you are sure that is the case, then at least I know how to compute it :) thanks again. $\endgroup$
    – z8080
    Commented Dec 4, 2017 at 12:27

1 Answer 1

2
$\begingroup$

By convention, $z$ is sometimes used to denote (or at least connote) quantiles from a standard normal distribution. With that in mind, you can see how "$z$-scores" is used for standardized data (even though standardizing is a linear transformation that will not make non-normal data normally distributed). The idea would be that if you assume your data are already normally distributed, standardizing them converts that into a standard normal distribution.
$$ z = \frac{X-\text{mean}}{\text{sd}} $$ What you are calling "$z({\rm proportion})$ is a different transformation (it might be better written as $\Phi^{-1} ({\rm proportion})$) that is also based on the assumption that the underlying data are normally distributed (since it's a latent variable, the 'standard' doesn't mean anything here—the mean and SD could be anything, calling them $0$ and $1$ is just a convenience). Thus, if we are willing to assume the underlying distribution is normal, we can pass the observed proportion backwards through the standard normal CDF to determine what the underlying quantile must have been:
$$ z = \Phi^{-1} ({\rm proportion}) $$ Thus, although these two transformations both connote a normal distribution, they are used in different contexts on different types of data. Moreover, the reason for these transformations being applied also differs. Standardizing a dataset is a linear transformation, and doesn't have much of an effect on anything. A reason why it might be done is to make the data more intuitive, if the units are unfamiliar. ('What is 475 IUs? Is that high or low? Oh, is that patient 1.5 SDs above the mean, I see.') On the other hand, when working with signal detections, you can convert the proportions of hits and false alarms into the clarity of the signal ($d'$) and the bias of the detector to say 'yes'. This is valid under the assumption that the latent distribution is normal, and it speaks more directly to what you want to know than the original proportions did.

Response to edit:
What is missed by using simple differences in proportions is that the base rates for those three sets of responses differ, and so the detectability is conflated with the bias. It's sloppy analysis, TBH.

Here are some references for further reading:

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.