Causal Inference After Feature Selection

Question

I am interested in this forum's thoughts concerning the use of LASSO for feature selection in a high dimensional dataset and subsequent OLS regression to adjust for confounding on the most frequently selected variables (I'm using 100 random draws). I'm aware that feature selection does not take into account the causal relationship of the selected variables. Therefore, I want to use OLS to adjust for potential confounding after feature selection. Are there potential issues that may arise? Has this been done in the literature before? Citations appreciated.

Answer 1

Data mining for potential predictors and causal inference don't go together well. Few problems may arise:

Identification problems:

• Confoundness: If some unobserved common causes are still not included in the data, the estimators are still biased.

• Bad control: Including too many variables is harmful. You can include colliders and unwanted mediators. This helps predictions, but harm causal inference badly. Estimators may be biased in all magnitudes and directions, especially after something as LASSO.

• Mismeasurement: If the variables are not measured perfectly, as they appear in the DGP (real world), their estimators may be biased towards zero (attenuation bias). However if you mismeasure control variables, they do not fully control for potential confoundness.

• Sampling issues: Unless the sample is 'representative' you may run into unwanted conditioning on colliders.

Estimation problems:

• False significancy: Whatever significance level you choose, that much of false rejections of nulls you will have. In theory you can try to adjust p-values by Bonferroni (or others) procedures, but this means the loss of power by exclusion of some previously significant predictors.

• Functional form: Unless the functional relationship between variables is perfectly described in model, issues similar to mismeasurement arise. It is hard to deal with it when you perform mass scanning of the data.

In theory, with some work, you can try to automatically solve estimation problems, but not the identification ones. This group of problems requires careful inquiry about DGP, and possibly some additional assumptions about how the data was generated.

---

EDIT:

Similar points make Hernan and Robins (2020) in chapter 18 of their book.

In 18.2 they emphasise the role of bias-inducing variables (what I referred to as bad control). Then they argue, that the decision wether to control for a variable must be based on the information outside the data. And therefore this decision can not be made by any automated procedures thet rely exclusively on statistical associations.

They also point out, that this problem is already deeply studied. They provide criticism, introduction to some basic solutions and additional citations.

---

Hernán MA, Robins JM (2020). Causal Inference: What If. Boca Raton: Chapman & Hall/CRC.