How do I calculate d' from experimental data?

Question

In Signal Detection Theory, d' is defined by the z-scores of Hits and False Alarms:

d' = z(Hits) - z(False Alarms).

Say the task is to detect if a certain object is present in a series of pictures, and for one particular subject we have a hit rate of 0.9 and a false alarm rate of 0.2.

Now, the z-score would be typically calculated by $\frac{X-\mu}{\sigma}$. The stimulus is as described however presence/absence of an object (without any changes in stimulus intensity). It is hence not clear to me what $\mu$ and $\sigma$ could be - or do we simply assume a standard normal distribution to calculate the z-scores?

Answer 1

Proportions (fractions of tested case) are typically described via binomial distribution:

You try $n$ times and observe $k$ successes with probability $p$ for each of your tries.

$$\Pr(X=k)={\binom {n}{k}}p^{k}(1-p)^{n-k}$$

Expectation and variance are:

$$\operatorname {E} (X)=np$$ $$\operatorname {Var} (X)=np(1-p)$$

Now, when measuring your hit and false alarm rates* that is measuring such a binomial $p$.
The point estimate for that based on observing $k$ successes in $n$ trials is $\hat p = \frac{k}{n}$, and its variance is $\operatorname{Var} (\frac{k}{n}) = \frac{p (1-p)}{n}$ (note that last $p$s here are the unknown true probability).

Normal approximation

For sufficiently large $n$, the binomial distribution will be approximately normal with variance $\frac{p (1-p)}{n}$, and that happens faster for $p$ close to 0.5.

This approximation is usually considered OK for a single proportion if both $np$ and $n (1-p)$ are larger than 5 or 10. In your case, this should be met for both hits and false alarms.

Difference between proportions

Fleiss et. al., Statistical Methods for Rates and Proportions (Wiley) in chapter 3 section "method II" gives the z-score for the difference as:

$$z = \frac{|p_2 - p_1| - \frac{1}{2}\left(\frac{1}{n_1} + \frac{1}{n_2}\right)}{\sqrt{\bar p (1 - \bar p) \left(\frac{1}{n_1} + \frac{1}{n_2}\right)}}$$

Here, $p_2$ and $n_2$ would refer to your hits and $p_1$ and $n_1$ to the false alarms.

• In the context of null hypothesis testing, the null hyothesis here is $p_2 = p_1$. $\bar p$ is the hypothetical equal probability for both, you can calculate its estimate from the fourfold table. It is also a weighted mean of $p_1$ and $p_2$ (according to the sample sizes).

• $-\frac{1}{2}\left(\frac{1}{n_1} + \frac{1}{n_2}\right)$ is a so-called contnuity correction. It improves the approximation of a discrete distribution (counting events) by a continuous distribution.

• there exist other approximations for the variance (denominator), and exact calculations. See e.g. the book I linked above for a starting point.

---

* the term rate is somewhat misleading here (although that's what they are often called): strictly speaking, a rate would describe the number of events within a given time frame (looking for that will get you to Poisson distributions). Here, we have number of events as a fraction of a given number of tests which would be a quote. I'll use proportion or fraction of tested cases to avoid confusion.

Answer 2

In classical Signal Detection Theory, the assumption is that both signal and noise stem from a normal distribution (typically a standard deviations equal 1 for both distributions, and a mean of 0 for the noise distribution is assumed).

Given hit and false alarm rates from the experimental data, one can calculate the z-scores based on a standard normal distribution. This is where the formula:

d' = z(Hits) - z(False Alarms)

stems from. If you look at the examples of the book cited below, you see that you can calculate the z-scores straight from these hit and false alarm rates without further transformation.

The reason is this: You will get always the same z-scores given a particular probability (in the experimental case rates, that is).

So, one has typically rates, z-scoring does not require knowledge of the params of the normal distribution associated with the signal (e.g. z(0.5) is always 0, no matter what mean and standard deviation the underlying gaussian had).

Example: Assume you have a normal distribution with $\mu=100, \sigma=15$. The cumulative density of 0.747 will correspond to a value of 110 for this distribution. Standardizing will lead to $\frac{110-100}{15}$ = 0.66...

But this is exactly norminv(0.747).

For further details, I recommend this book of Macmillan and Creelman.