I have a binary dependent variable and a continuous independent variable. When I apply logistic regression to build the prediction model, I use p-value to know whether the independent variable is a significant predictor for the independent variable. I would like to repeat the same study using two other ML techniques: classification and regression trees and random forest. For each of these techniques how to know whether the prediction model is significant?

  • 4
    $\begingroup$ Welcome to Cross Validated! Some of this is going to depend on what exactly you mean by “significant”. If you mean that the software printout gives a p-value that either is or is not below $0.05$ (or some other value), then you see that you don’t get such a printout. Consequently, I suggest that you consider what you learn from that p-value that you want to learn about your tree-based model. For instance, the overall F-test of a linear regression tests your model against a baseline model that only has an intercept. The logic is that, if you can’t beat that, you’re not doing much of value. $\endgroup$
    – Dave
    Mar 8 at 19:55
  • 4
    $\begingroup$ Welcome to CV. Could you explain what you mean by a "significant" model? In the logistic regression a parameter estimate may be "significant" insofar as there is some evidence the true parameter is nonzero. What is the analog of that for a CART or RF, in your view? $\endgroup$
    – whuber
    Mar 8 at 19:55
  • $\begingroup$ Look at "variable importance" measures for ML models. $\endgroup$
    – usεr11852
    Mar 8 at 20:48
  • 1
    $\begingroup$ Please register &/or merge your accounts (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. $\endgroup$ Mar 9 at 12:13


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.