2
$\begingroup$

In Signal Detection Theory, d' is defined by the z-scores of Hits and False Alarms:

d' = z(Hits) - z(False Alarms).

Say the task is to detect if a certain object is present in a series of pictures, and for one particular subject we have a hit rate of 0.9 and a false alarm rate of 0.2.

Now, the z-score would be typically calculated by $\frac{X-\mu}{\sigma}$. The stimulus is as described however presence/absence of an object (without any changes in stimulus intensity). It is hence not clear to me what $\mu$ and $\sigma$ could be - or do we simply assume a standard normal distribution to calculate the z-scores?

$\endgroup$
  • $\begingroup$ I think your problem is that you are confusing two meanings of z(). See the Wikipedia entry and see if that helps en.wikipedia.org/wiki/Sensitivity_index $\endgroup$ – mdewey Jan 28 '19 at 16:22
  • $\begingroup$ Thanks! Which one of the answers below are correct? $\endgroup$ – user24544 Jan 28 '19 at 17:38
1
$\begingroup$

Proportions (fractions of tested case) are typically described via binomial distribution:

You try $n$ times and observe $k$ successes with probability $p$ for each of your tries.

$$\Pr(X=k)={\binom {n}{k}}p^{k}(1-p)^{n-k}$$

Expectation and variance are:

$$\operatorname {E} (X)=np$$ $$\operatorname {Var} (X)=np(1-p)$$

Now, when measuring your hit and false alarm rates* that is measuring such a binomial $p$.
The point estimate for that based on observing $k$ successes in $n$ trials is $\hat p = \frac{k}{n}$, and its variance is $\operatorname{Var} (\frac{k}{n}) = \frac{p (1-p)}{n}$ (note that last $p$s here are the unknown true probability).

Normal approximation

For sufficiently large $n$, the binomial distribution will be approximately normal with variance $\frac{p (1-p)}{n}$, and that happens faster for $p$ close to 0.5.

This approximation is usually considered OK for a single proportion if both $np$ and $n (1-p)$ are larger than 5 or 10. In your case, this should be met for both hits and false alarms.

Difference between proportions

Fleiss et. al., Statistical Methods for Rates and Proportions (Wiley) in chapter 3 section "method II" gives the z-score for the difference as:

$$z = \frac{|p_2 - p_1| - \frac{1}{2}\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}{\sqrt{\bar p (1 - \bar p) \left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$$

Here, $p_2$ and $n_2$ would refer to your hits and $p_1$ and $n_1$ to the false alarms.

  • In the context of null hypothesis testing, the null hyothesis here is $p_2 = p_1$. $\bar p$ is the hypothetical equal probability for both, you can calculate its estimate from the fourfold table. It is also a weighted mean of $p_1$ and $p_2$ (according to the sample sizes).

  • $-\frac{1}{2}\left(\frac{1}{n_1} + \frac{1}{n_2}\right)$ is a so-called contnuity correction. It improves the approximation of a discrete distribution (counting events) by a continuous distribution.

  • there exist other approximations for the variance (denominator), and exact calculations. See e.g. the book I linked above for a starting point.


* the term rate is somewhat misleading here (although that's what they are often called): strictly speaking, a rate would describe the number of events within a given time frame (looking for that will get you to Poisson distributions). Here, we have number of events as a fraction of a given number of tests which would be a quote. I'll use proportion or fraction of tested cases to avoid confusion.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Not sure if this answers @Testguest's question. Given Hit and False Alarm rates - one calculates the z-scores by using a standard normal distribution - here these probabilities (Hit, FA) can be "inverted" (norminv or such a function) to get said z-scores. Hence, the particular underlying distribution of signal and noise should be irrelevant. $\endgroup$ – Pugl Jan 26 '19 at 20:04
  • $\begingroup$ @Pegah: I'm sorry, but I don't get your argument as to why the underlying distribution is irrelevant (as opposed to: find out how do a suitable normal approximation). If it turns out that there's a well-known approximation even for the difference expressed in standard deviation units, I'd say, all the better. But, while I'm familiar with looking at the difference between signal and noise level and also expressing such a difference in standard deviation units, my field uses a different terminology. So maybe you should add an answer as well? $\endgroup$ – cbeleites unhappy with SX Jan 28 '19 at 11:31
0
$\begingroup$

In classical Signal Detection Theory, the assumption is that both signal and noise stem from a normal distribution (typically a standard deviations equal 1 for both distributions, and a mean of 0 for the noise distribution is assumed).

Given hit and false alarm rates from the experimental data, one can calculate the z-scores based on a standard normal distribution. This is where the formula:

d' = z(Hits) - z(False Alarms)

stems from. If you look at the examples of the book cited below, you see that you can calculate the z-scores straight from these hit and false alarm rates without further transformation.

The reason is this: You will get always the same z-scores given a particular probability (in the experimental case rates, that is).

So, one has typically rates, z-scoring does not require knowledge of the params of the normal distribution associated with the signal (e.g. z(0.5) is always 0, no matter what mean and standard deviation the underlying gaussian had).

Example: Assume you have a normal distribution with $\mu=100, \sigma=15$. The cumulative density of 0.747 will correspond to a value of 110 for this distribution. Standardizing will lead to $\frac{110-100}{15}$ = 0.66...

But this is exactly norminv(0.747).

For further details, I recommend this book of Macmillan and Creelman.

| cite | improve this answer | |
$\endgroup$
  • $\begingroup$ Sure. But what are the $z$s of a hit rate of $\frac{4}{5}$ and $\frac{80}{100}$, respectively? Also (I don't know Signal Detection Theory in captital letters, I just know about signal and noise in chemistry and some adjacient fields), does it really assume standard normal distribution or does it ask for signal and noise to be transformed into standard normal random variables? $\endgroup$ – cbeleites unhappy with SX Jan 28 '19 at 14:19
  • $\begingroup$ It does not assume a standard normal distribution for noise and signal. But for the same probabilities (!, not absolute values), the z-scores will always be the same. I.e. for any normal distribution of the signal, if you have a hit rate of 0.8 (or 4/5) this will correspond to a z-score of 0.84. Take different normal distributions, calculate their value for prob = 0.8, then standardize this value and you will always get the mentioned z-score. Hence, you don't need an explicit transformation. And by the way, it doesn't matter if the hit rate is 0.5 or 4/5, the principle always holds. $\endgroup$ – Pugl Jan 28 '19 at 15:04

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy