8
$\begingroup$

Statistical Learning and its results are currently pervasive in Social Sciences. A couple of months ago, Guido Imbens said: "LASSO is the new OLS".

I studied Machine Learning a little bit, and I know that its main goal is prediction. I also agree with Leo Breiman's distinction between two cultures of statistics. So, from my point of view, causality is opposed to prediction to some extent.

Considering that sciences usually try to identify and understand causal relations, is machine learning useful for this goal? In particular, what are the advantages of LASSO for causal analysis?

Are there any researchers (and papers) addressing those questions?

Improve this question
$\endgroup$
5
  • 2
    $\begingroup$ Well, OLS will not produce estimates of causal effects very often, so if LASSO is to replace OLS, it does not have the "burden" of discovering causal relations. That said, have a look at this page for some recent research in econometrics on causal effects and sparse methods: mit.edu/~vchern $\endgroup$
    – Christoph Hanck
    Feb 4 '16 at 5:03
  • 1
    $\begingroup$ For me the more natural distinction here would be that by Shmueli ("To Explain or to Predict", 2010) rather than Breiman's, but perhaps Breiman's distinction is also fine. $\endgroup$
    – Richard Hardy
    Feb 4 '16 at 7:30
  • $\begingroup$ @ChristophHanck . Well, you're right. But the point is: OLS has been employed for estimating causal effects a lot. For example, 'Mostly Harmless Econometrics' address several subjects related to this. Therefore, if it is possible with OLS, why not with LASSO? Anyway, Thank you for the reference. $\endgroup$
    – Guilherme Duarte
    Feb 4 '16 at 11:00
  • $\begingroup$ @RichardHardy You're completely right. I know this paper. I just mentioned Breiman, because I thought it would be easier to explain. $\endgroup$
    – Guilherme Duarte
    Feb 4 '16 at 11:01
  • 2
    $\begingroup$ I don't disagree there: in cases in which OLS can be used to estimate casual effects, I do not see why lasso should not also be applicable $\endgroup$
    – Christoph Hanck
    Feb 4 '16 at 11:06
2
$\begingroup$

I don't know all of them, I'm sure, so I hope no one will mind if we do this wiki-style.

One important one though is that the LASSO is biased (source, Wasserman in lecture, sorry), which while acceptable in prediction, is a problem in causal inference. If you want causality, you probably want it for Science, so you're not just trying to estimate the most useful parameters (which happen strangely to predict well), you're trying to estimate the TRUE(!) parameters.

Improve this answer
$\endgroup$
14
  • $\begingroup$ Good answer! Actually if you have bias, it's a big deal for causal estimates. But maybe LASSO could be employed preliminarily in a more complete procedure to assess causality. $\endgroup$
    – Guilherme Duarte
    Feb 4 '16 at 17:00
  • $\begingroup$ Perhaps! That's why I'm eager to have other people chime in. $\endgroup$
    – one_observation
    Feb 4 '16 at 17:02
  • 1
    $\begingroup$ @GuilhermeDuarte, It is the overall error that matters, not bias. Under square loss we care about MSE, and that equals Bias$^2$ + Variance. Lasso may deliver a good tradeoff with relatively small MSE despite some bias and as such should be more useful for causal analysis than unbiased estimation with high MSE. The real problem with lasso is that it is difficult to get confidence intervals for it; currently that is an active research area. $\endgroup$
    – Richard Hardy
    Mar 13 '17 at 19:17
  • $\begingroup$ @RichardHardy sorry, you mean when we care about causality, we shouldn't be concerned about bias, but with the MSE? This is not entirely clear to me $\endgroup$
    – Guilherme Duarte
    Mar 14 '17 at 13:20
  • 1
    $\begingroup$ @GuilhermeDuarte, just as in prediction, in causality we need precise estimates of model coefficients. Precision can be measured in terms of absolute error, squared error, etc., but not bias. For example, you can have low bias and high estimation error at the same time. So looking at bias you would think you are doing fine, but that would be misleading as the estimation error (absolute, squared or whichever) is high. It is estimation error, not bias that matters when you consider effect sizes, statistical significance etc. in causal inference. $\endgroup$
    – Richard Hardy
    Mar 14 '17 at 13:43

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.