Comparing correlations for two variables in different overlapping groups

Question

I have a sample where I have applied five (wildly differing) diagnostic definitions for the same disease. For example, 50% of people who are disease cases under diagnostic definition 1 are not disease cases under diagnostic definition 2, and 10% of people who meet definition 2 are not considered cases under definition 1. Definition 1 is currently the most widely-used "standard", all other definitions will be compared to definition 1.

I want to compare the correlation between two variables that are measured for each person (call them X and Y) and see if cor(X,Y) among people who are disease cases under definition 1 differs from cor(X,Y) among people who are disease cases under definition 2 (keeping in mind that some people meet both definitions and will be in both groups). There are several pairs of variables I want to do this with - depending on the specific variables, I may be using Pearson, Spearman, or tetrachoric correlations.

After restricting the sample to "people who meet at least one of the five diagnostic definitions" my sample size is approximately 80,000, and all diagnostic definitions produce at least 30,000 cases.

My first thought about how to approach this would be to use bootstrapping:

1. Sample with replacement using the whole sample of "80,000 people who meet at least one diagnostic definition".
2. For each of the five diagnostic definitions, calculate cor(X,Y) using observations in the bootstrap sample that qualified under that definition.
3. Use a Fisher transformation on the resulting correlations so subtraction will be appropriate.
4. For each of diagnostic definitions 2-5, subtract the correlation for that group from the correlation from the definition 1 cases. (Using the Fisher-transformed correlations from the current bootstrap iteration.)
5. Repeat steps 1-4 thousands of times.
6. Take min(percent of differences > 0,percent of differences < 0), and multiply that number by 2 to get a two-tailed test of whether the difference between the correlations is not equal to 0.

Is this procedure appropriate? Will it do what I think it does (test for a difference in correlations while accounting for overlap between the two groups that produced the correlations)? Is there something else I should consider using instead?

Answer 1

I liked your approach! Have you thought about the fact that bootstrapping people who satisfy both definitions may inflate the cor(X,Y) ? Maybe a paired bootstrapping approach would be more effective in considering this potential violation of the independence assumption inherent to the Bootstrap. This is one hypothesis:

1. Sample with replacement from the full dataset (80,000 individuals).

2. For each bootstrap iteration, identify the individuals who qualify for both Definitions 1 and 2.

3. Calculate cor(X,Y) for these individuals under Definitions 1 and 2.

4. Store the difference in these correlations.