I'm building a regression model but the metric I care about most is a bit different from tradition measures. That is, I want to see if actual(test_instance1) > actual (test_instance2), what is the probability that predicted(test_instance1) > predicted(test_instance2). I'm not aware of any existing metric in regression world to capture this. How how can calculate this?
$\begingroup$
$\endgroup$
5
-
$\begingroup$ This can be estimated as a function of the two actual values (or sometimes as a function of their difference). What form of regression and what regression fitting procedure are you interested in? $\endgroup$– whuber ♦Commented Jan 2, 2019 at 22:32
-
$\begingroup$ I'm doing a random forrest regression in python, how can I define this function then? $\endgroup$– HHHCommented Jan 2, 2019 at 22:36
-
$\begingroup$ You can't make this comparison at all with a random forest, because it assumes no probabilistic model in the first place. $\endgroup$– whuber ♦Commented Jan 3, 2019 at 15:28
-
$\begingroup$ which models supports this? what if I switch to GBM? What's is the methodology behind calculating this measure? $\endgroup$– HHHCommented Jan 3, 2019 at 15:45
-
$\begingroup$ In order for "probability" to make any sense, you need to posit a probabilistic model. The form of the model will indicate how to compute (or estimate) the probabilities you are asking about. $\endgroup$– whuber ♦Commented Jan 3, 2019 at 16:14
Add a comment
|