Standard regression output suggests which covariates are significant and similarly standard random forest output suggests which are relatively important for the response.

We might be able to see the important relationships visually, but for large N or high dimensional data, how can we determine the set of covariates that has had the maximum impact on the response for a given observation? For example, if $Y = x_1 + x_2 + X_3$, then $Y$ can equal $2$ for many combinations of $x_1$, $x_2$ and $x_3$ values.

By comparison, if we build a CART tree then we can traverse the tree from the bottom up to get the cause of various covariates on the response. But it is not possible for random forest or regression. Are there any analytical methods to assess this?

I am providing practical example – Everyday “stats.stackexchange” gets many requests and based many parameters (IVs – count of SME available for that particular subject, backlog in that queue, weekday/holiday,…) the time to response (DV – assume - 1 hour after question post) are calculated/predicted.

On any particular day, the time to response is 5 hours (say 5 times of expected mean time of response) then question comes why today response time is 5 hours and what factors (IVs) has contributed. This is what, I am trying to understand

  • 2
    $\begingroup$ I am not really sure it is possible to establish causality in this context. Besides, your question is not clear with this limited information. $\endgroup$ – T.E.G. - Reinstate Monica Jan 25 '17 at 9:12
  • $\begingroup$ If we build tree "rpart" then we can traverse bottom-up to get the cause of various IVs on DV. Although it is not possible for RF and LM. Is there any Analytics to take care of Fish bone flow $\endgroup$ – Shiv Jan 25 '17 at 10:26
  • 3
    $\begingroup$ Traversing a simple tree from bottom up to the root node does not give "causes" of IVs on the DV in any meaningful sense. That is an illusion. $\endgroup$ – gung - Reinstate Monica Jan 25 '17 at 15:11
  • $\begingroup$ There is also a literature on relative importance in regression although I am not sure at the moment whether that is what your question is about. $\endgroup$ – mdewey Jan 25 '17 at 15:15
  • $\begingroup$ Although based on a mistaken premise, this question seems clear enough to me. I'm voting to leave open. $\endgroup$ – gung - Reinstate Monica Jan 25 '17 at 15:19

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.