0
$\begingroup$

Correlation attenuation: Question about Spearman’s correction

Disclaimer: I’m not a statistician, so please don’t be surprised about ignorance and/or outrageously stupid faults in reasoning. But do let me know, please!

I’m interested in the expected attenuation of a hypothesized correlation by imperfect measures. My motivation is to take this into account for power calculations. For a hypothesized correlation rxy between two properties, what is the expected observed correlation rx’y’ between two imperfect measures of these properties? The imperfectness of these measures is given by their validity (rx’x and ry’y; their correlation with the property they aim to measure), which in turn can be estimated by their re-test reliabilities (rx'1x'2 and ry'1y'2; which really are upper bounds of their validity).

Running some simulations I stumbled upon a simple pattern:

rx’y’ = rxy * rx'x * ry'y

which – I thought - surely has been described before. Googleing a bit, I hit on Spearman’s ‘disattenuation’ (https://en.wikipedia.org/wiki/Correction_for_attenuation, https://www.jstor.org/stable/1412159).

As far as I see, this is looking at the same thing from the opposite end. Spearman was interested in estimating rxy from rx’y’, by correcting for the attenuation effect of rx'1x'2 and ry'1y'2. My problem is that his equation is different from my results:

rxy = rx’y’/ sqrt(rx'1x'2*ry'1y'2)

which implies

rx’y’ = rxy * sqrt(rx'1x'2*ry'1y'2)

Basically, the effect of attenuation in my simulations is the square of that given by Spearman. So I went back and double-checked that my simulations do what I thought they would do. Checking the properties of the simulated data, everything seems to be in order: the simulated ‘true’ values x and y correlate with rxy; the measurements x’ correlate with x by rxx, y’ and y correlate by ryy. However, rx’y’ is equal to rxy attenuated by the product of rx’x and ry’y, rather than the square root of that product.

Thinking about this discrepancy, I also noticed that Spearman’s attenuation formula goes against my intuition. Take the simplest case: Two properties that are perfectly correlated (rxy = 1), a perfect measure of the criterion (ryy = 1) and a measure of the predictor that has limited reliability (rxx <1). For this case, my simulations would suggest

rx’y’= 1*1*rx'x = rx'x,

whereas Spearman would predict

rx’y’ = 1*sqrt(1*rx'1x'2) = sqrt(rx'1x'2).

I don’t understand how Spearman’s solution makes sense. It implies that the observed correlation of x’ with y’ will be higher than the correlation of x’ with itself (rx'1x'2). How can that possibly be true? It seems obvious to me, that if rx'1x'2 is the only limiting factor for rx’y’ (because everything else is perfectly correlated), rx’y’ should be equal to rx'1x'2.

So, where do I go wrong? How does the square root in Spearman’s equation make sense? What could explain this discrepancy between his equation and the simulation results?

Thanks in advance for any help – much appreciated!

$\endgroup$
0
$\begingroup$

OK, thanks to clever facebook friends I think I got it now. The crux is the difference between retest reliability (e.g. rx'1x'2) and validity (rx'x). rxx in Spearman's formula refers to rx'1x'2. In the original question I used rxx to refer to the validity rx'x, as if it were the same thing (notations now edited for clarity).

I did not think of this as a problem, because I assumed reliability to be an upper bound estimate of validity. That turns out to be wrong however. The cap that reliability puts on validity (rx'x_max) is rx'x_max = sqrt(rx'1x'2).

That is because re-test reliability is affected by twice the measurement error, due to the repetition. The correlation between measures and true values is only affected by one instance of the measurement error.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.