# How to Make Meaningful Conclusions?

I recently appeared for an Interview for my college and I was asked the following question. The Interviewer said that this question was a Data Science question. He asked the same question to a friend of mine as well.

The question-

Suppose 7.5% of the population has a certain Bone Disease. During COVID pandemic you go to a hospital and see the records. 25% of the COVID Infected patients also had the Bone Disease. Can we say for sure if the Bone Disease is a symptom of COVID-19?

My Reponse-

I said No, and explained it as it's not necessary that COVID-19 is causing these symptoms, it could very well be possible that the 7.5% of the country's population which already had the disease is more susceptible to the virus due to lowered immunity. Hence making conclusions is not possible.

Then the interviewer asked me How can we be sure if it is a symptom or not?

I replied saying we can go to more Hospitals, collect more data and see if it correlates everywhere.

The Interviewer then said If we have the same results everywhere will you conclude it's a symptom?

I had no good answer but I replied that Just correlation of data is not sufficient, we also need to check if the people who have COVID-19 had the bone disease prior to getting infected or not. See if that percentage also correlates and stuff.

Here he stopped questioning however I couldn't judge If I was right or wrong.

I am in Grade-12 so I have no experience in Data Science as such. I do know a fair bit of statistics however I have never solved such questions. Can someone provide me insights on how to solve such questions and make meaningful conclusions?

I have asked the same question on Data Science SE however i noticed the other questions there were quite different so I wasn't sure if this question is appropriate there. On Maths SE I was told it is appropriate for Stats SE as well so i'm posting it here too

• They’re doing technical interviews for college admissions?
– Dave
Jul 29, 2020 at 1:58
• Most colleges here have entrance exams and no interviews but this college is gaining quite a reputation here for very renowned faculty members. The course is a bachelors + masters research oriented course in Computer Sciences so that might explain the Data Science question @Dave They also asked general stuff and not just technical questions Jul 29, 2020 at 2:06
• Getting at your question, the probability of having Covid and the vine disease is $0.25$. Can you relate $P(\text{COVID} \cap \text{Bone Disease})$ to a conditional probability? (Yes.) I see this as a pure probability problem.
– Dave
Jul 29, 2020 at 2:23
• Can you elaborate a bit more @Dave and how do I make conclusions from the conditional probability? If i use the relation P(AnB)=P(A|B)(P(B)) How do i get the P(A|B) part? If i am not wrong it means Probability of getting COVID given the person has a Bone Disease right? So how can we be sure he got the Bone Disease before or after getting COVID? Jul 29, 2020 at 2:28
• @Dave I disagree that this is a pure probability -- a la sophmopre year stats class -- play. This sounds more like causal inference, no? How might the joint density of covid and bone disease inform people that bone disease is a symptom (unless the interviewer failed to mention that the people in the hospitals are a simple random sample and that all people are equally likely to contract covid). Any answer seems like it may be subject to OP's original answer. Jul 31, 2020 at 4:30

This is a causal inference question. "Symptomhood" is a necessarily causal concept, because the disease causes the symptom. The question amounts to "Does COVID-19 cause changes in the risk of the bone disease?" This is an important type of question for data scientists to be able to answer because many data science questions are of the form "Does X cause Y?", e.g., Does this new web interface improve engagement?, Does this new medicine reduce the risk of death?, Does this new policy increase employment?, etc. Causal inference is normally taught at the graduate level, though some argue that it should be a part of training at all levels of education.

One of the challenges in answering causal questions is to distinguish association (i.e., correlation) due to a causal relationship and association due to some other reason. For example, there may be an association between attending a private college and adult earnings, but that association might be at least partly due to the fact that coming from a wealthy family improves your probability of attending a private college and would increase your adult earning whether you went to a private college or not. When variables cause both the predictor of interest and the outcome of interest, there is said to be "confounding", and the simple association between the predictor and outcome doesn't necessarily represent a causal relationship.

There are a number of ways to address confounding, but many of them are taught only at the graduate level. Often, the best way to address confounding is to run an experiment: randomly assign people to the levels of the predictor and then measure their outcome. This addresses confounding because now the predictor and outcome share no common causes. In this example, unfortunately, and many others, randomly assigning some people to get COVID-19 and comparing the risk of bone disease between the exposed and unexposed is unethical.

The interviewer probably wanted you to use critical thinking to think about other ways you could address confounding. There are many such ways (again, mostly taught at the graduate level), but the simplest is stratification. Create strata (i.e., groups) of people who didn't have bone disease prior to March 2020 and who, given what we know about how bone disease develops, had similar risks of bone disease (e.g., same age, same family history of bone disease, same comorbidities, etc.). Within each stratum of bone disease risk, you could see whether those in the stratum who had COVID-19 were more likely to later develop the bone disease. Within strata, you can't say that some people were just more likely to get the bone disease anyway; within strata, everyone is equally likely to get the bone disease. The only meaningful way individuals within strata differ is whether they have COVID-19 or not. So, you couldn't attribute differences in the probability of getting the bone disease to anything other than differences in COVID-19 status.

Despite being the most fundamental way to address confounding without an experiment, this is fairly sophisticated, so I would be surprised if the interviewer expected you to say something like this. But they probably hoped you would say something in this direction, i.e., to address the factors that cause both COVID-19 and bone disease and would explain why the two appear correlated. By eliminating alternative explanations for the observed association, you can eventually be reasonably sure about whether the observed association is causal, i.e., whether the bone disease is caused by, and therefore is a symptom of, COVID-19.

• Nice explanation thanks a lot Jul 31, 2020 at 4:47
• Glad it was helpful. Also want to shout out @Demetri Pananos who posted basically the same thing.
– Noah
Jul 31, 2020 at 4:59
• Thanks for mentioning I've thanked him in the comments Jul 31, 2020 at 5:01
• @Noah I began to doubt myself. +1 for the excellent answer Jul 31, 2020 at 12:03