Better methodologies to make causal recommendations from correlated data?

Question

I work as a data scientist at a SAAS company. We have an outcome variable, Y, that we consider "success" for our customers. We have a bunch of additional outcome variables X1, X2, X3 that come chronologically before Y and are highly correlated with Y. The goal is to identify which Xs to prioritize and develop interventions for (i.e. advertise those features to increase adoption), with the goal of increasing Y.

The relationship between the Xs and Y is not clear. While there may be some causal element (doing X causes Y), a more reasonable model might be a latent structure, where some unobserved Z causes X and Y to go up in parallel.

The proposed approach has been to just fit some model to measure the correlation between the Xs and Y (logistic regression, random forest), and prioritize the Xs that most strongly correlate with Y as the variables to create interventions for. I worry this will waste a lot of time and effort chasing correlations that will have no impact on on Y, but short of setting up experiments (which there's no appetite for), I can't think of a better analytic method to make recommendations about the importance of various Xs. How would you approach this problem?

Answer 1

I guess one thing to think about is why you think the correlations between X and Y won't be causal, in the sense that manipulating X wouldn't lead to increases in Y. The most common case would be confounding: that an unobserved variable is a cause of both X and Y leading you to see a spurious relationship.

If that's the case there are two main strategies. If you can think of what U might be and can measure it then (with some assumptions) you could include this in your model and be more confident the prediction from X to Y is causal.

If that's not possible, another strategy is to think of a third variable, Z, which you think predicts X, but does NOT predict Y directly. So a causal diagram would look like

Z -> X -> Y
U -> X
U -> Y

If you can find a variable Z that satisfies this condition (and it's really important that Z doesn't have a direct relation to Y, and also isn't related to U) then you can use instrumental variable methods to get a causal method for X on Y. Another name for this is a natural experiment.

One classic example in economics is the attempt to estimate the effect of army service on earnings. There are lots of potential confounders of this relationship, but during the Vietnam war army service was determined by lottery, and (from memory) the first letter of your surname was used to determine how early you got called up. See Angrist and Imbens' Mostly Harmless Econometrics for this and other nice examples. Their treatment of fitting these models is quite focussed on an approach called, 2-stage-least squares, which is perhaps slightly older and less flexible than using path analysis (e.g. with lavaan in R) but the principles are the same.

It's hard to say more without knowing the substantive area, but another excellent introduction to this field is Judea Pearl's Book of Why. It's a popularisation of his much denser book, Causality which has been very influential in the field, specifically in the use of causal diagrams to determine whether and how to make causal inferences from observational data.