(Sorry for the George Orwell/Animal Farm reference)
I am an MBA student taking courses in Statistics. Over the Thanksgiving weekend, we have an informal assignment - we have to find publicly available data on two "concepts" and make a small presentation on the possible correlations between these two "concepts" and if the correlation might be "spurious" or not.
I chose to explore the relationship of countries between the "Number of Doctors Per 1000 People" and the "Life Expectancy" of people in that country. On the surface, this sounds like there might be some correlation - as a brutish example, imagine if there are few doctors (relative to the population) - assuming if not seeing a doctor can kill you, countries with fewer doctors would result in sitting in the waiting/emergency room for longer periods of times (assuming that the prevalence and severity of medical conditions are similar across different countries, as well as the quality of doctors) ... ultimately resulting in more people dying in general and lowering the life expectancy in countries with smaller doctor/people ratios. Although I have greatly and comically oversimplified this situation, I would hope that there is some grain of truth in this oversimplification and it would be reflected within the data.
I found data on the ratio of doctors/population in different countries over here (https://data.oecd.org/healthres/doctors.htm, https://data.oecd.org/healthres/doctors.htm - I copy/pasted this into Excel) and I found data on the life expectancy over here (https://data.worldbank.org/indicator/SP.DYN.LE00.IN). I am learning how to use R - I downloaded both of these datasets and made this quick graph to investigate the correlation:
# NOTE: This is not a Doctoral Thesis, but a very informal learning assignment where the data doesn't need to be fully accurate - please take it easy on me.. thank you! # PART 1: DOCTOR DATA doctors = read.csv("doctors_country.csv") #PART 2: LIFE EXPEC DATA #chop off the first few rows in excel to make the data manipulation easier life_e = read.csv("life_expec.csv") life_e = data.frame(life_e$Country.Name, life_e$X2020) colnames(life_e) <- "Country" colnames(life_e) <- "Life_Expec" #PART 3: JOIN FILE graph_file = merge(x = life_e, y = doctors, by = "Country", all.y = TRUE) graph_file = na.omit(graph_file) # PART 4: PLOT GRAPH par(mfrow=c(1,2)) plot(graph_file$Life_Expec, graph_file$Doctors, main = "Correlation Between Life Expectancy and Number of Doctors Per 100 People (2020)" , xlab = "Life Expectancy (Years)", ylab = "Number of Doctors Per 1000 People") text(graph_file$Life_Expec, graph_file$Doctors, labels=graph_file$Country)
Based on this (very hurried) analysis that I performed, it would appear that there is somewhat of a linear correlation between these two concepts (we can also calculate the Correlation Coefficient to find out how strong, i.e. Rho Value). It appears that the more doctors there are relative to the population, the life expectancy tends to also increase. Hurray!
But here is my question: How Do I Know That This Relationship Is Not Spurious? Maybe for generations, there is a secret undiscovered vitamin in the water of Norway and Austria that makes people live longer - and this increase in life expectancy has nothing to do with the high ratio of doctors/people!!??
In a previous question (Does Correlation "Sometimes" Imply Causality?), I started learning about "Causality vs Correlation" and found out that it is very difficult to ascertain the presence of Causality.
However in this question (i.e. Doctors vs Life Expectancy), is there truly a way to determine whether or not this correlation is spurious or not? It seems to me that there is no straightforward answer to this question - such a question can only be answered by rigorous experiments, subject matter knowledge and science. I am guessing that extensive reviews of the healthcare systems, exogeneous and endogenous factors, life style habits, epidemiological patterns, review of academic intuitions ... all would have to be comprehensively studied over decades to find out if in fact this doctor/life expectancy correlation is spurious or non-spurious. And this extensive research will likely not be finished by next Wednesday (i.e. the date of my informal presentation).
Thus, we have no choice but to interpret our results under the following framework : Although philosophically, all correlations might be partly spurious - some correlations are likely to be far more spurious than other correlations (e.g. divorce rate in Maine/vowels in the name of hurricanes vs life expectancy/number of doctors). Although nothing can be ever be conclusively determined (e.g. I can not be 100% sure that a rhinoceros is standing outside my door - but I can still be more relatively certain that a pink rhinoceros is not standing outside my door, and even more relatively certain that a pink a rhinoceros riding a unicycle is not standing outside my door - but I can be pretty damn certain). However, we can not twiddle our thumbs indefinitely waiting on a universe of delusional and highly improbable events to manifest and risk jeopardizing and delay taking important decisions that can better mankind (e.g. a pharmaceutical drug performed very well in many clinical trials and has demonstrated the ability to save lives with minimum side effects - but we will never market this drug because technically all test patients might have just gotten lucky). Thus, using domain specific knowledge and rigorous statistical methodologies can help us attribute a level of confidence in the interim and provide reasonable beliefs if a specific correlation is spurious or not as we learn more about our data with each passing day and proceed to make important data based decisions to hopefully improve society.
Can someone again confirm if I am "out to lunch" (i.e. crazy) - or is my understanding somewhat correct?