What is an appropriate statistical test of significance of whether the result of one medical exam has affected the participation in a second exam?

Question

I have a number of participants of a medical exam. After getting their result, they then have the option to have another, more detailed, exam. Visually, it seems that if a participant receives a high risk score, they are much more likely to decide to get the second exam. What is a recommended statistical test to calculate a p value (significant at p < 0.05 or p = 0.082, for example), to be able to say, "ok, I'm confident this is not just random variance").

Here's the data:

Exam 1 Result	Exam 1 no. of participants	Exam 2 no. of participants
Low risk	65	17
Medium risk	12	4
High risk	16	10
Total	93	31

I've looked a chi-square test for independence, but it doesn't seem to make any distinction of the different rows, and the examples i've seen all use distinct categories, not "follow-up" exams.

Thank you for any help you can provide :D

Answer 1

You were generally on the right track with $\chi^2$, but the devil is in the details.

If you were thinking of a 3x2 contingency matrix (3 groups: risk level. 2 outcomes: got a 2nd exam, did not), you are correct that it will not discren that the groups have ordinal scale, nor will it tell you which cells are disproportionate, etc. It is an omnibus test, that tells you that the proportions are (or not) different between the 3 groups.

Now, what you can do is run a set of 2x2 contingency matrices, comparing pairs of groups: low vs. medium, medium vs. high, and low vs. high. You can then run these tests with a single tail (since you are trying to prove that the higher the risk level, the higher the likelihood to get the 2nd exam), which increases your power.

You should not really run $\chi^2$ tests, but instead a Fisher-exact tests. This is because your medium group is too small for the normal approximation to be valid (only 4 chose a 2nd exam).

So, for example, comparing low to high, you contingency table is $\begin{matrix} 17 & 10 \\48 & 6\\\end{matrix}$. The single sided p-value (B>A) is 0.0079 (Fisher-exact) or 0.0029 via $\chi^2$.

For low vs. medium, the p-value is 0.422 (Fisher-exact). The reason here is that the proportions are too similar (26.2% vs. 33.33.%) for the sample sizes (but you would need much larger sample sizes; e.g. if you had 10 times more subjects in both groups, with the same proportions, the p-value (0.0671) would still not be significant (at the 0.05 level).

Similarly, for medium vs. high, the p-value is .126. The reason here is that the 2 sample sizes (12 amd 16) are too small (even though the observed proportions are very different: 33.3 vs. 62.5%, almost twice as many). Here, if you just doubled the sample sizes for both groups (assuming same proportions), then the p-value would go to 0.029.

So to address this issue of small sample sizes, you can either collect more data, or decide to aggregate the medium group with either the high or low group (p-value becomes significant at the .05 level in both cases). Or be satisfied to conclude that the medium group can not be statistically distinguished from the other 2 groups from this data...

Answer 2

If you have access to the individual (low, medium, high) risk scores for each participant at each time point (Exam 1, Exam 2), you can use correlation (and/or regression) analysis for ordinal variables to address this question.