Yes, I think there can always be some unexplored factor that --- had we evaluated that factor --- would have changed our interpretation of the results. That's just a reality of imperfect knowledge. And particularly problematic in observational studies like the one described where the observations are not balanced. (That is, where there are unequal numbers of each sex in each class).
But we should take some solace in the fact that we have some opportunities to assess our data to the best of our abilities.
For this example, the odds ratio for the first table is 1.007, suggesting the difference in survival rate between the two classes is so tiny that we likely would not have considered it interesting. That is, the survival rate for each class is essentially 24%.
The upshot here is that I think this example is less an example of a paradox where the trend reverses, than an example of seeing nothing interesting in the first table, but finding something interesting when more information is added in the second table.
It's only when we have the information in the second table that we get some sense of the factors affecting survival.
Because the underlying question is about what we can conclude about the effect of Class on survival rate, I'll use logistic regression to answer this question.
##### Table 2 #####
Data = read.table(header=T, text="
Class Sex Survive NotSurvive
Third M 75 387
Third F 76 89
Crew M 192 670
Crew F 20 3
")
Trials = cbind(Data$Survive, Data$NotSurvive)
model = glm(Trials ~ Class + Sex + Class:Sex,
data = Data,
family = binomial(link="logit"))
library(car)
Anova(model)
### Analysis of Deviance Table (Type II tests)
###
### Response: Trials
### LR Chisq Df Pr(>Chisq)
### Class 13.510 1 0.0002373 ***
### Sex 88.568 1 < 2.2e-16 ***
### Class:Sex 8.502 1 0.0035472 **
Note that the interaction of Class and Sex is significant, suggesting that this is the effect that we should be paying attention to.
In the results below, prob is the probability calculated in the table in the question.
library(emmeans)
emmeans(model, ~ Class:Sex, type="response")
### Class Sex prob SE df asymp.LCL asymp.UCL
### Crew F 0.8695652 0.07022340 Inf 0.6645495 0.9573281
### Third F 0.4606061 0.03880395 Inf 0.3860325 0.5369860
### Crew M 0.2227378 0.01417187 Inf 0.1961989 0.2517422
### Third M 0.1623377 0.01715628 Inf 0.1314483 0.198824
We can also use estimated marginal means to estimate what the survival rate for each of classes would be had the sexes been balanced in each class. Below, we see that in fact, the survival in Crew is meaningfully and statistically higher.
This is a different conclusion than we would have come to from using the information in the first table only.
emmeans(model, ~ Class, type="response")
### Class prob SE df asymp.LCL asymp.UCL
### Crew 0.5802181 0.07605615 Inf 0.4284069 0.7182285
### Third 0.2891697 0.02063485 Inf 0.2504569 0.3312222
The addition of the information on sex has improved our understanding, but, still, there could always be some other important factor we have failed to measure that would have changed our interpretation.