I have a binary dependent variable and a continuous independent variable. When I apply logistic regression to build the prediction model, I use p-value to know whether the independent variable is a significant predictor for the independent variable. I would like to repeat the same study using two other ML techniques: classification and regression trees and random forest. For each of these techniques how to know whether the prediction model is significant?
4
-
4$\begingroup$ Welcome to Cross Validated! Some of this is going to depend on what exactly you mean by “significant”. If you mean that the software printout gives a p-value that either is or is not below $0.05$ (or some other value), then you see that you don’t get such a printout. Consequently, I suggest that you consider what you learn from that p-value that you want to learn about your tree-based model. For instance, the overall F-test of a linear regression tests your model against a baseline model that only has an intercept. The logic is that, if you can’t beat that, you’re not doing much of value. $\endgroup$– DaveMar 8 at 19:55
-
4$\begingroup$ Welcome to CV. Could you explain what you mean by a "significant" model? In the logistic regression a parameter estimate may be "significant" insofar as there is some evidence the true parameter is nonzero. What is the analog of that for a CART or RF, in your view? $\endgroup$– whuber ♦Mar 8 at 19:55
-
$\begingroup$ Look at "variable importance" measures for ML models. $\endgroup$– usεr11852Mar 8 at 20:48
-
1$\begingroup$ Please register &/or merge your accounts (you can find information on how to do this in the My Account section of our help center), then you will be able to edit & comment on your own question. $\endgroup$– kjetil b halvorsen ♦Mar 9 at 12:13
Add a comment
|