2
$\begingroup$

In reviewing my notes about making causal inferences under the selection on the observables identification strategy, I reviewed some pieces that make critiques against contemporary strategies in observational settings that use regression adjustment. I'll use Samii 2016 as an example although the arguments made in this paper are fairly widespread from what I can observe.

Samii (2016) critiques conventional regression-based studies on numerous grounds:

  • Psuedo-generality problem: users of regression adjustment rarely make clear the weights generated from their regression analysis, using descriptive statements about the spatial-temporal coverage of their data set to implicitly suggest that their regression models weight the data similarly.
  • Manipulating regression model specifications to achieve a favorable result (I would also throw in here the critique that regression is sensitive to functional form specifications).
  • Poor control variable selection: Many users of regression adjustment may include "bad controls" (Cinelli et al. 2022)
  • Evaluating multiple treatment effects in a single regression equation
  • Barring an experiment, causal inference is difficult

Here is where my question begins. I was taught that matching strategies and inverse probability weighting (IPW) are natural "upgrades" over standard regression adjustment strategies. Certainly, I think explaining the logic of these methods is easier than explaining how regression weights are generated, but what methodological advantages, specifically for making causal inferences do matching/weighting hold over regression adjustment? It seems to me that, for the five points made above, matching/weighting are on an equal footing with standard regression adjustment if bad practices in standard regression adjustment studies are addressed.

For example:

  • Users of regression adjustment can make their weights generated from a regression model clear and transparent. With that being said, users of matching/IPW can also fail to be transparent concerning the cases kept post-matching and IPW-generated weights.
  • One can manipulate the variables chosen to match on/generate propensity scores on to achieve a favorable outcome.
  • Likewise, one can choose "bad" variables to match on/generate propensity scores from.
  • Users of regression adjustment do not inherently have to follow the practice of attempting to evaluate multiple treatment effects in a single regression model.
  • Matching/IPW suffer from the potential of unobserved confounding just as much as regression adjustment does.

In summary, I feel as if most of the critiques against regression adjustment are levied towards the culture of regression adjustment rather than the method itself. However, I completely acknowledge that I may be confused on an important point here. I acknowledge the value of matching/IPW as alternative strategies under selection on the observables, but I fail to articulate why these methods should preform any better than regression adjustment at estimating causal effects (assuming the use of regression adjustment has kicked some of its associated bad practices).

$\endgroup$
0

1 Answer 1

1
$\begingroup$

This is an interesting question, which was discussed several times here and elsewhere.

I can link some reference which I found very useful when dealing with this question: A nice paper on the topic, and a very interesting thread on datamethods. Some other will certainly give more details on the topic but I think this is a very good starting point.

Overall I tend to support the idea that a well constructed (and perhaps validated) covariate-adjusted regression model performs well in most cases.

$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.