6
$\begingroup$

In signal detection theory, people often use $d'$ to assess performance. Apart from the fact that $d'$ is in $z$ units (units of measurement transformed to standard deviation units, i.e., $z$ scores), making it comparable regardless of the original units of measurement. I can't see what the advantage in analysing $d'$ instead of proportion correct is.

Don't both account for bias? Would they follow the same shaped ROC curves?

Improve this question
$\endgroup$
2
  • 4
    $\begingroup$ Could you please clarify 'follow the same shaped ROC curves'? $\endgroup$
    – Trisoloriansunscreen
    Mar 9, 2014 at 11:40
  • 2
    $\begingroup$ There is precisely no context here. Also, it is rare that notation like this is constant across disciplines, so explaining what the notation means would increase the possible value of this question to others. $\endgroup$
    – Nick Cox
    Mar 9, 2014 at 11:41

1 Answer 1

Reset to default
7
$\begingroup$

D' is a measure of sensitivity, whereas proportion correct is affected by both sensitivity and bias.

In the special case of two balanced classes (same number of signal and noise trials) and zero bias, D' is monotonically mapped to proportion correct, offering no additional insight. However, if the two classes are not completely balanced or the bias isn't zero, the two measures can considerably depart. Consider these two examples:

  1. A dataset with 70% signal trials and 30% noise trials. An observer / classifier always responding 'signal' will have 0.7 proportion correct but zero D'.

  2. A dataset with balanced classes and a classifier with D'=1. A zero bias would produce a maximal proportion correct and any increase / decrease in the bias are excepted to decrease the proportion correct (think of the case of extreme biases).

Improve this answer
$\endgroup$
13
  • $\begingroup$ That's really helpful. I now realise that D prime is necessary where there are uneven amounts of signal and noise trials....previously I was only thinking about equal signal and noise trial situations. e.g. with 50% signal and 50% noise, an observer always responding signal will have .5 correct and .5 incorrect. proportion correct = .5 and d=zero. But if there are 70% signal trials and 30% noise trials, you start to get advantages by increasing your willingness to response 'signal'. proportion correct = .7 But d' is still comes out at zero: d = zhits - zFA = 0.524 - 0.524 = 0 $\endgroup$
    – user41270
    Mar 9, 2014 at 12:18
  • $\begingroup$ i guess in this situation for an observer to increase their d' they would need to start responding above 70% correct. $\endgroup$
    – user41270
    Mar 9, 2014 at 12:24
  • $\begingroup$ consider also the second example - even amounts of signal and noise trials. If there's zero sensitivity, bias won't change the proportion correct, but if sensitivity>0, bias does have an effect. $\endgroup$
    – Trisoloriansunscreen
    Mar 9, 2014 at 12:46
  • $\begingroup$ Regarding the 70/30 example, you can have D'>0 and proportion correct<0.7 if there's some sensitivity but the bias is suboptimal. $\endgroup$
    – Trisoloriansunscreen
    Mar 9, 2014 at 12:48
  • 1
    $\begingroup$ @Mecasickle, thank you for your attention to this answer! In my 70/30 example, Hit rate would be 1 and false alarm rate would be 1, hence d'=0. False alarm rate is p(responding 'signal'|no signal). $\endgroup$
    – Trisoloriansunscreen
    Nov 21, 2016 at 7:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.