I work as a data scientist at a SAAS company. We have an outcome variable, Y, that we consider "success" for our customers. We have a bunch of additional outcome variables X1, X2, X3 that come chronologically before Y and are highly correlated with Y. The goal is to identify which Xs to prioritize and develop interventions for (i.e. advertise those features to increase adoption), with the goal of increasing Y.
The relationship between the Xs and Y is not clear. While there may be some causal element (doing X causes Y), a more reasonable model might be a latent structure, where some unobserved Z causes X and Y to go up in parallel.
The proposed approach has been to just fit some model to measure the correlation between the Xs and Y (logistic regression, random forest), and prioritize the Xs that most strongly correlate with Y as the variables to create interventions for. I worry this will waste a lot of time and effort chasing correlations that will have no impact on on Y, but short of setting up experiments (which there's no appetite for), I can't think of a better analytic method to make recommendations about the importance of various Xs. How would you approach this problem?