What is the equation for random forest?

Question

I need an equation for random forest so that I can score fresh data I receive every week, based on beta estimates I got after building model using this ensemble methodology.

Every week I do not want to build random forest model again and then score it using following commands in Python. Because I have build 15 such models and I need to score them every week. So easiest way is to get the equation (betas I already have) and score.

# importing the ensemble libraries
from `numpy` import `genfromtxt`
from `sklearn.metrics` import `classification_report`

#importing the dataset

dataset = genfromtxt(open('~//win_5050_6oct.csv','r'))[1:]  

target = [x[1] for x in dataset]
train = [x[2:] for x in dataset]
test = genfromtxt(open('~\\win_act_6oct.csv','r'))[1:]

val = [x[2:] for x in test]
y_act = [x[1] for x in test]
Gst_i=[x[0] for x in test]

#building random forest model

rf = RandomForestClassifier(n_estimators=100)
rf.fit(train, target)

Answer 1

First of all,the equation you are looking for is not possible for random forest. This is because the nature of random forest algorithm inherently leads to destruction of any simple mathematical representation.

Random forest works by building decision trees & then aggregating them & hence the Beta values have no counterpart in random forest. Though you do get the 'Variable Importance /Gini Index' values for the forest, which can be used for making sense of the model but not as a multiplication factor.

Now to answer your question, from your code snippet it seems that you have stored you model as rf. To use this model for prediction, you can simply call the predict method in python associated with the random forest class.

use:

prediction = rf.predict(test)

This will give you the predictions for you new data (test here) based on the model rf. The predict method won't build a new model, it'll use the model rf to use for prediction on new data.

For a deeper understanding behind how a random forest model is built,you can check out the chapter on Random Forests from Elements of Statistical Learning

For a simplified working example, you should check out yhatq article here:

http://blog.yhathq.com/posts/comparing-random-forests-in-python-and-r.html

Answer 2

You could write the equation in terms of indicator functions for a single decision tree. Consider the following simple example:

1. $y = 1$ if $x < 5$
2. $y = 2$ if $5 \leq x \leq 10$
3. $y = 3$ if $x > 10$

Then you could express the function as

$$y = 1 \times I(x < 5) + 2 \times I(5 \leq x \leq 10) + 3 \times I(x > 10)$$

This wouldn't generalize to ensemble methods like Random Forest, though, as noted in the comments above. It also doesn't accomplish anything that is not already expressed in pretty much any decision tree implementation; it's just a different expression of the same information.

Answer 3

No, it doesn't make any sense to use the measure of importance for the explanatory variables in the same way as regression coefficients.

Trees are non-parametric models, for which the prediction is usually a constant (such as the mean for each region, when $y$ in continuous, or the most common class, in the case of classification).

The regression coefficients have an important role when dealing with parametric models, as they describe the parameters. In the case of linear regression, for example, the parameter $\mu = X\beta$, where $\beta$ is the vector of regression coefficients. For trees, in the other hand, the $"\mu"$ (quoted, because there is no official $\mu$ in a non-parametric model) is just a constant, such as the mean for the regions formed by the explanatory variables. Then, the variables have nothing to do with the prediction, besides creating the regions. They are responsible only the for configuration of the set of rules that will create the regions, for which the prediction is the mean of the response variable, $y$, in it. The measures of importance are then related to the regions, not the predictions.