66
votes
Famous easy to understand examples of a confounding variable invalidating a study
Coffee Drinking & Lung Cancer
My favorite example is that supposedly, "coffee drinkers have a greater risk of lung cancer", despite most coffee drinkers... well... drinking coffee, rather than ...
61
votes
Accepted
Why is Average Treatment Effect different from Average Treatment effect on the Treated?
The Average Treatment Effect (ATE) and the Average Treatment Effect on Treated (ATT) are commonly defined across the different groups of individuals. In addition, ATE and ATT are often different ...
- 46k
29
votes
Is Propensity Score Matching a "MUST" for Scientific Studies?
Propensity score methods are one type of method used to adjust for confounding. There are several other methods that rely on different assumptions. Some of the most popular include difference-in-...
- 36.8k
24
votes
Accepted
What are the pros and cons of using mahalanobis distance instead of propensity scores in matching
Mahalanobis distance matching (MDM) and propensity score matching (PSM) are methods of doing the same thing, which is to find a subset of control units similar to treated units to arrive at a balanced ...
- 36.8k
19
votes
Famous easy to understand examples of a confounding variable invalidating a study
You might want to introduce Simpson's Paradox.
The first example that page is the UC Berkeley gender bias case where it was thought that there was gender bias (towards males) in admissions when ...
18
votes
Famous easy to understand examples of a confounding variable invalidating a study
Power Lines and Cancer
After an initial study finding a link between living next to high-voltage transmission lines and cancer, follow-up studies found that when you include income in the model the ...
13
votes
Why is Average Treatment Effect different from Average Treatment effect on the Treated?
ATE is the average treatment effect, and ATT is the average treatment effect on the treated.
The ATT is the effect of the treatment actually applied. Medical studies typically use the ATT as the ...
- 277
12
votes
Is Propensity Score Matching a "MUST" for Scientific Studies?
As Alexis pointed out, propensity score matching (PSM) is one of many tools we have in causal inference. Another one is Inverse Probability Weighted Estimator (IPWE). You can also use causal discovery ...
- 1,063
11
votes
Accepted
In causal inference in statistics, how do you interpret the consistency assumption in mathematical terms?
Let me use $X$ for the treatment, $Y$ for the observed outcome and $Y(x)$ for the potential outcome under $X = x$.
Consistency means that for an individual $i$, his observed outcome $Y_i$ when $X_i =...
- 12.7k
11
votes
Accepted
How exactly to evaluate Treatment effect after Matching?
The documentation for Matching is sadly fairly incomplete, leaving what it does quite mysterious. What is clear is that it takes a different approach from Stuart (...
- 36.8k
11
votes
Famous easy to understand examples of a confounding variable invalidating a study
Consider the following examples. I am not sure they are necessarily very famous but they help to demonstrate the potential negative effects of confounding variables.
Say one is studying the ...
10
votes
Can age ever be confounded if it is the independent variable in an observational study?
First image below: With variables 'health' and 'wealth' you describe a situation with mediators. Age has a causal effect on health and wealth, and these in their turn have an effect on happiness.
...
- 86.5k
9
votes
Famous easy to understand examples of a confounding variable invalidating a study
There was one about diet that looked at diet in different countries and concluded that meat caused all sorts of problems (e.g. heart disease), but failed to account for the average lifespan in each ...
8
votes
Accepted
Adjusting for baseline as covariate in observational studies
Adding baseline as a covariate is statistically acceptable - or in fact advisable - in observational studies, as well as RCTs. It is typically just not sufficient to ensure valid inference, and ...
- 35.2k
8
votes
Accepted
Recurring problem with retrospective data collection study designs I'm seeing
You are right that this is a very common scenario in medical research.
"I should note that these studies are not meant to invent a new method of treatment or change protocols, they are used to see ...
- 65.8k
7
votes
advantages and disadvantages of IPTW vs propensity score matching?
The choice between propensity score matching and weighting seems to be a widely debated topic among statistical sholars. Some thoughts, after having read through many papers of infuriated ...
- 221
7
votes
Famous easy to understand examples of a confounding variable invalidating a study
I'm not sure it entirely counts as a confounding variable so much as confounding situations, but animals' abilities to find their way through a maze may qualify.
As described in this ScienceDirect ...
7
votes
Accepted
Simpson's paradox in Freedman, Pisani and Purves book
This is explained on same page (p. 19):
Technical note. Table 2 is hard to read because it compares 12 admission rates. A statistician might summarize table 2 by computing one overall admissions rate ...
- 14.3k
6
votes
Accepted
Conditional treatment effect and average treatment effect under no unmeasured confounders (ignorability)
CATE = ATE when there is no treatment effect modification. If you conduct a study in a rural area and attempt to generalize to a population of rural people, then conduct another study in an urban area ...
- 36.8k
6
votes
advantages and disadvantages of IPTW vs propensity score matching?
Despite some similarities, propensity score matching (PSM) and inverse probability of treatment weighting (IPTW) behave differently, mainly because matching selects some cases/controls and discards ...
- 2,622
6
votes
Accepted
What happens if I try to match on outcome probability? (rather than propensity score/treatment probability)
The "probability of outcome" is indeed called the prognostic score. I recommend Hansen (2008) for a full account of prognostic scores and their value in causal inference. Like propensity ...
- 36.8k
6
votes
Why is ANCOVA not appropriate for modelling post-intervention outcome, controlling for baseline
This is explained very clearly in Lüdtke and Robitzsch (2020). I'll briefly summarize their arguments below but the paper is clear and easy to read, and I recommend you read it.
Essentially, the ...
- 36.8k
6
votes
Can age ever be confounded if it is the independent variable in an observational study?
Observational studies adjust for age all the time! I would not be surprised if age was the most common adjustment variable in observational studies.
If age is the exposure (as in both examples of @...
- 31.2k
6
votes
Accepted
Sufficient Sample Size for Three-way Interaction from Observational Data (2x2xContinuous)
Simply to avoid overfitting, the rough rule of thumb for regression with a binary outcome is at least 15 members of the minority class per coefficient that you try to estimate. Section 4.4 of Frank ...
- 102k
5
votes
In propensity score matching, should a variable used in exact matching also be used in the model?
Yes, we can/is recommended to use a variable $x$ that we used for matching in our final model. The matching itself can also have different steps as here, both exact and then PSM. Using multiple ...
- 46k
5
votes
Accepted
Study design question: What's the best design to assess harm of an exposure?
I start with my methodological thoughts and I offer some footnotes with thoughts that came to my mind on the ethics. Take both of these with a large grain of salt, because we know very little on your ...
- 35.2k
5
votes
Famous easy to understand examples of a confounding variable invalidating a study
There was a great study of mobile phone use and brain cancer. Most people with a lateral brain cancer, when asked which hand they hold their phone in, answer the diseased side. This seemed to show ...
5
votes
Accepted
Analyzing a counterfactual in observational studies?
In observational studies, when randomization is not an option (everyone is treated), it is necessary to estimate a counterfactual of what would had happened in the absence of treatment.
It is not ...
- 6,454
5
votes
Accepted
Basic understanding of control variables in observational studies
Factors, whose only connection with the considered variables is that they influence only the dependent variable, in particular, have no connection with the independent variable, will not cause bias to ...
- 11.2k
5
votes
Accepted
Should I use independent- or propensity score matching?
Use whatever matching method yields the best balance. Sometimes that will be propensity score matching and sometimes it will be some other form of matching. I recommend genetic matching, which ...
- 36.8k
Only top scored, non community-wiki answers of a minimum length are eligible