In reviewing my notes about making causal inferences under the selection on the observables identification strategy, I reviewed some pieces that make critiques against contemporary strategies in observational settings that use regression adjustment. I'll use Samii 2016 as an example although the arguments made in this paper are fairly widespread from what I can observe.
Samii (2016) critiques conventional regression-based studies on numerous grounds:
- Psuedo-generality problem: users of regression adjustment rarely make clear the weights generated from their regression analysis, using descriptive statements about the spatial-temporal coverage of their data set to implicitly suggest that their regression models weight the data similarly.
- Manipulating regression model specifications to achieve a favorable result (I would also throw in here the critique that regression is sensitive to functional form specifications).
- Poor control variable selection: Many users of regression adjustment may include "bad controls" (Cinelli et al. 2022)
- Evaluating multiple treatment effects in a single regression equation
- Barring an experiment, causal inference is difficult
Here is where my question begins. I was taught that matching strategies and inverse probability weighting (IPW) are natural "upgrades" over standard regression adjustment strategies. Certainly, I think explaining the logic of these methods is easier than explaining how regression weights are generated, but what methodological advantages, specifically for making causal inferences do matching/weighting hold over regression adjustment? It seems to me that, for the five points made above, matching/weighting are on an equal footing with standard regression adjustment if bad practices in standard regression adjustment studies are addressed.
- Users of regression adjustment can make their weights generated from a regression model clear and transparent. With that being said, users of matching/IPW can also fail to be transparent concerning the cases kept post-matching and IPW-generated weights.
- One can manipulate the variables chosen to match on/generate propensity scores on to achieve a favorable outcome.
- Likewise, one can choose "bad" variables to match on/generate propensity scores from.
- Users of regression adjustment do not inherently have to follow the practice of attempting to evaluate multiple treatment effects in a single regression model.
- Matching/IPW suffer from the potential of unobserved confounding just as much as regression adjustment does.
In summary, I feel as if most of the critiques against regression adjustment are levied towards the culture of regression adjustment rather than the method itself. However, I completely acknowledge that I may be confused on an important point here. I acknowledge the value of matching/IPW as alternative strategies under selection on the observables, but I fail to articulate why these methods should preform any better than regression adjustment at estimating causal effects (assuming the use of regression adjustment has kicked some of its associated bad practices).