2
$\begingroup$

I need an equation for random forest so that I can score fresh data I receive every week, based on beta estimates I got after building model using this ensemble methodology.

Every week I do not want to build random forest model again and then score it using following commands in Python. Because I have build 15 such models and I need to score them every week. So easiest way is to get the equation (betas I already have) and score.

# importing the ensemble libraries
from `numpy` import `genfromtxt`
from `sklearn.metrics` import `classification_report`

#importing the dataset

dataset = genfromtxt(open('~//win_5050_6oct.csv','r'))[1:]  

target = [x[1] for x in dataset]
train = [x[2:] for x in dataset]
test = genfromtxt(open('~\\win_act_6oct.csv','r'))[1:]

val = [x[2:] for x in test]
y_act = [x[1] for x in test]
Gst_i=[x[0] for x in test]

#building random forest model

rf = RandomForestClassifier(n_estimators=100)
rf.fit(train, target)
$\endgroup$
  • $\begingroup$ This thread provides another "view" into the relationship between RF feature importances and linear models. stats.stackexchange.com/questions/164048/… $\endgroup$ – Sycorax Jun 21 '18 at 19:05
  • $\begingroup$ LIME -- uc-r.github.io/lime $\endgroup$ – EngrStudent Jun 21 '18 at 20:41
  • 1
    $\begingroup$ @EngrStudent: Only if the local approximation is ridiculously good (and obviously LIME works only for a single point). $\endgroup$ – usεr11852 Jun 21 '18 at 20:44
  • $\begingroup$ @usεr11852 - the point of Lime is to back out the "physics". This is where human in the loop can be value returning. I have found several times that what a random forest could do a good job at, the right glm could also do, and the RF can lead to the glm. The glm is "cleaner". $\endgroup$ – EngrStudent Jun 25 '18 at 3:06
  • 1
    $\begingroup$ @EngrStudent: I do not disagree. That being said: "a small parcel defined by the local radius of accuracy for the truncated Taylor approximation" sounds a wee bit convoluted... LIME is mostly designed to offer a simple visualisation of the classifier's decision criteria to an audience that usually does not have a ML background. What are you referring to here? :) $\endgroup$ – usεr11852 Aug 5 '18 at 23:31
7
$\begingroup$

First of all,the equation you are looking for is not possible for random forest. This is because the nature of random forest algorithm inherently leads to destruction of any simple mathematical representation.

Random forest works by building decision trees & then aggregating them & hence the Beta values have no counterpart in random forest. Though you do get the 'Variable Importance /Gini Index' values for the forest, which can be used for making sense of the model but not as a multiplication factor.

Now to answer your question, from your code snippet it seems that you have stored you model as rf. To use this model for prediction, you can simply call the predict method in python associated with the random forest class.

use:

prediction = rf.predict(test)

This will give you the predictions for you new data (test here) based on the model rf. The predict method won't build a new model, it'll use the model rf to use for prediction on new data.

For a deeper understanding behind how a random forest model is built,you can check out the chapter on Random Forests from Elements of Statistical Learning

For a simplified working example, you should check out yhatq article here:

http://blog.yhathq.com/posts/comparing-random-forests-in-python-and-r.html

$\endgroup$
  • $\begingroup$ "the nature of random forest algorithm inherently leads to destruction of any simple mathematical representation", what do you mean by this statement? $\endgroup$ – baxx Mar 27 at 16:05
3
$\begingroup$

No, it doesn't make any sense to use the measure of importance for the explanatory variables in the same way as regression coefficients.

Trees are non-parametric models, for which the prediction is usually a constant (such as the mean for each region, when $y$ in continuous, or the most common class, in the case of classification).

The regression coefficients have an important role when dealing with parametric models, as they describe the parameters. In the case of linear regression, for example, the parameter $\mu = X\beta$, where $\beta$ is the vector of regression coefficients. For trees, in the other hand, the $"\mu"$ (quoted, because there is no official $\mu$ in a non-parametric model) is just a constant, such as the mean for the regions formed by the explanatory variables. Then, the variables have nothing to do with the prediction, besides creating the regions. They are responsible only the for configuration of the set of rules that will create the regions, for which the prediction is the mean of the response variable, $y$, in it. The measures of importance are then related to the regions, not the predictions.

$\endgroup$
2
$\begingroup$

You could write the equation in terms of indicator functions for a single decision tree. Consider the following simple example:

  1. $y = 1$ if $x < 5$
  2. $y = 2$ if $5 \leq x \leq 10$
  3. $y = 3$ if $x > 10$

Then you could express the function as

$$y = 1 \times I(x < 5) + 2 \times I(5 \leq x \leq 10) + 3 \times I(x > 10)$$

This wouldn't generalize to ensemble methods like Random Forest, though, as noted in the comments above. It also doesn't accomplish anything that is not already expressed in pretty much any decision tree implementation; it's just a different expression of the same information.

$\endgroup$
  • $\begingroup$ Well, if we are going down the route of numerous indicator functions we might as well write out all $K$ trees and scale the contributions of each individual tree by $\frac{1}{K}$ before aggregating them together... Which is as practical as sorting pennies wearing boxing gloves but you know... "plausible". $\endgroup$ – usεr11852 Jun 21 '18 at 20:42
  • $\begingroup$ I agree -- you'll notice I didn't argue for this as a particularly useful or practical solution. But knowing neither the background nor the needs of the asker, I thought showing what such an equation could look like might still improve understanding. $\endgroup$ – djlid Jun 22 '18 at 11:50
  • $\begingroup$ Sure thing, we are good! I did not say it was wrong. Actually +1 cause probably it is useful indeed. $\endgroup$ – usεr11852 Jun 22 '18 at 17:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.