Is Random Forest the best way for calculating continuous variable using 5 categorical variable

Question

I have 5 categorical variables. I binarize them into buckets and then assign 1 for the matched and 0 for others.

Next, I run a RandomForestRegressor using scikit-learn. But, my accuracy is only 18% on the test dataset. I don't know what to improve upon ? Should we use Random Forest where the predictor variables are categorical in nature?

categorical = [u'vendor_id', u'part_id', u'ship_to_location_id',
       u'bill_to_location_id', u'carrier_number']

    df_x.iloc[0].tolist
    Out[652]: 
    <bound method Series.tolist of 
vendor_id_435835                           1
vendor_id_437307                           0
vendor_id_422290                           0
vendor_id_421933                           0
vendor_id_425392                           0
vendor_id_421725                           0
vendor_id_421961                           0
vendor_id_437323                           0

The output variable is integer. It is the lag time for each vendor, part_id, source country,destination country and carrier chosen (air, ship, road etc). There is a relationship for sure, but how to effectively predict the delivery lag days which is my output variable. Sample values are as below:

df_y
Out[655]: 
0         4
1         1
2         1
3         9
4         1
5        58
6         3
7         7
8         5
9         5
10        7

To check the values, wrote a block of code. As pointed out, accuracy_metrics is only capturing the exact matches.

count = 0
for i,val in enumerate(pred):
    if ((val + val * 0.1) == actual[i]):
        count += 1
    elif ((val - val * 0.1) == actual[i]):
        count += 1
    elif val == actual[i]:
        count += 1
    else:
        print ("actual:",actual[i],"predicted:",val)

EDIT 2: @Matthew Drury pointed out the MSE would be a better score to track.

import sklearn.metrics as sm
print ("MSE RandomForest:",sm.mean_squared_error(actual,pred))

Using this the RandomForest gave a very small score for MSE.

Answer 1

You say the response variable is an integer measuring the number of days for something to happen, but then measure the performance of your model using the accuracy.

Consequently, you are only counting a success if your model gets the answer exactly right. Is this really what you want? Is your model's prediction really worthless if you predict 10 lag days, but the reality is 9? Using accuracy, you are not crediting your model at all for being close, only when it is exactly, completely, and totally correct.

I would instead evaluate your model based on some measure of the difference between your prediction, and the true value. The average squared error, or average absolute error are two common choices.