I have a question about something that my statistics teacher said about the following problem:

There are two hospitals named Mercy and Hope in your town. You must choose one of these in which to undergo an operation. You decide to base your decision on the success of their surgical teams. Fortunately, under the new health plan, the hospitals give data on the success of their operations, broken down into five broad categories of operations. Suppose you get the following data for the two hospitals:

Mercy Hospital
Type         A    B      C    D      E    All
Operations  359  1836   299   2086  149  4729
Successful  292  1449   179   434   13   2366
Hope Hospital
Type          A   B  C   D   E   All
Operations   88 514 222 86  45   955
Successful   70 391 113 12  2    588


You notice that, in all types of operations, Mercy has a higher success rate than Hope, yet Hope has the highest overall success rate. Which hospital would you choose and why (choose two answers)?

A) Mercy; since I would go in for a specific operation, I want the hospital that has the best success rate for that operation.

B) Hope; since they do fewer operations in all categories, they are not "operation-happy" like Mercy.

C) Hope; this is an example of Simpson's paradox and we should always chose the "obvious" conclusion.

D) Mercy; looking at column E, Mercy clearly does more difficult surgeries and so is probably a better hospital.

E) Hope; it has the better overall success rate.

F) Mercy; this is an example of Simpson's paradox and we should always chose the opposite of the "obvious" conclusion.

My question isn't even about the occurrence of Simpson's paradox in this situation. My question is simply about the fact that my professor insists that A) and D) are the right answers instead of A) and F). He says,

"Because the success rate is so low for Type E surgeries,we can conclude that they are difficult and not just uncommon. Hence, Mercy probably has better equipment/doctors when compared to Hope."

I don't understand how he could imply on a statistical basis that he can tell that Mercy does "more difficult surgeries". It is obvious that Mercy has better success rate at type E surgeries, but why does that mean they do "more difficult surgeries". I think I am being screwed over by the wording of this problem and the professor isn't budging. Can someone please explain why I am wrong or how I can explain this to the professor?

• Oh wow, I'm sorry, you're totally right. I didn't really see that there was a SE site for statistical analysis. Thank you. – swiecki Jan 28 '12 at 21:39
• No need to be sorry. I was just alerting you to this fact in case you might happen to be unaware. You can click on the "flag" link and just ask it to be migrated there. It should happen fairly quickly. (+1) on the question, too, by the way. – cardinal Jan 28 '12 at 21:44
• I'm going to migrate this question to the statistics.SE site. There will be a link that appears below the question here that you can follow to the new location of your question. If you need help associating an account on statistics.SE, you can flag your question for moderator attention, and someone over there will help out. – Zev Chonoles Jan 29 '12 at 5:22

I think A and E aren't a good combination, because A says you should pick Mercy and E says you should pick Hope.

A and D have the virtue of advocating the same choice. But, lets examine the line of reasoning in D in further detail, since that seems to be the confusion. The probability of success for the surgeries follows the same ordering at both hospitals, with the A type being most likely to be successful and the E type being the least likely. If we collapse over (i.e., ignore) the hospitals, we can see that the marginal probability of success for the surgeries is:

Type     A     B     C     D     E     All
Prob   .81   .78   .56   .21   .08     .52


Because E is much less likely to be successful, it is reasonable to imagine that it is more difficult (although in the real world, other possibilities exist as well). We can extend that line of thinking to the other four types also. Now lets look at what proportion of each hospital's total surgeries are of each type:

Type     A     B     C     D     E
Mercy  .08   .39   .06   .44   .03
Hope   .09   .54   .23   .09   .05


What we notice here is that Hope tends to do more of the easier surgeries A-C (and especially B & C), and fewer of the harder surgeries like D. E is pretty uncommon in both hospitals, but, for what it's worth, Hope actually does a higher percentage. Nonetheless, the Simpson's Paradox effect is going to mostly be driven by B-D here (not actually column E as answer choice D suggested).

Simpson's Paradox occurs because the surgeries vary in difficulty (in general) and also because the N's differ. It is the differing base rates of the different types of surgeries that makes this counter-intuitive. What is happening would be easy to see if both hospitals did exactly the same number of each type of surgery. We can do that by simply calculating the success probabilities and multiplying by 100; this adjusts for the different frequencies:

Type     A     B     C     D     E     All
Mercy   81    79    60    21    09     250
Hope    80    76    51    14    04     225


Now, because both hospitals did 100 of each surgery (500 total), the answer is obvious: Mercy is the better hospital.

• +1 I was playing with pbinom in R while you answered this. :) – Michelle Jan 29 '12 at 7:12
• Oh dear, in looking over your answer, I realized that I made a slight mistake in providing details: I believe A) and F) to be the answer, not E) as it obviously does not match. Sorry about that. If you would be so kind as to leave another comment or answer addressing answer F), I would be more than happy to upvote it and of course accept this response. – swiecki Jan 30 '12 at 0:57
• So the disagreement is he says A & D, and you say A & F, is that right? If you're trying to convince him to give you points for your answer anyway, you could say that surgery E isn't the primary driver of the effect, as I show above. OTOH, F isn't really a good answer, it appeals to a recognition of the phenomenon without a solid understanding of it. Since there are only 3 choices that advocate for Mercy (the correct hospital) that leaves A & D. Moreover, surgery E is part of the effect even if it's not the biggest influence. I would've picked A & D, but the answers were poorly designed. – gung - Reinstate Monica Jan 30 '12 at 2:01
• +1 This is just about the clearest explanation I have come about of Simpson's paradox (thanks!). One very minor thing - in your last table I get a slightly different result for the last column, first row (github.com/RInterested/SIMULATIONS_and_PROOFS/blob/master/…) – Antoni Parellada Nov 15 '15 at 7:01
• @gung Ah! So you were referring to percentages, not the integers? – NNOX Apps Dec 25 '17 at 18:33

None of the answers are entirely baseless. But they ALL assume significant external knowledge and can't be taken to be correct strictly on the basis of the statistics.

A, B, D, and E all require assumptions about the factors the cause patients to choose one hospital over another; the process by which doctors and patients are matched up, the extent to which success rates are attributable to specific classes of operations vs. shared factors like ICU, and on an on.

In the real world we could legitimately consider many alternate factors such as the payment providers the hospital officially accepts, the socioeconomics and obesity rates of the neighborhood, whether this is a teaching hospital (in which case the success rate plummets when new interns arrive and we have to consider monthly mix), and on and on.

Obviously we can and do make reasonable assumptions about these factors, but without specifically addressing or excluding them from the problem, it's impossible to say if an answer is "right" or not.

@gung gave a very thorough answer, but there is one more reason why D is a correct answer to the question: Better hospitals do more of the difficult operations because they are better. That is, if a person comes into Hope Hospital for operation E (the hardest) they may send him/her to Mercy because they at Hope don't know how to do it.

This even happens in the real world, with the most difficult cases being sent to larger or more specialized hospitals.

• Isn't Operation E the hardest in the example? Also, in the problem, we know operation E is performed at both Hope and Mercy because we have data on them. – Jarad Aug 10 '16 at 21:26
• E is the hardest, my mistake, but while the two hospitals both do E, they don't do an equal proportion of E. That's part of the reason it's a paradox. – Peter Flom Aug 11 '16 at 15:12