1
$\begingroup$

I am using Random Forest to predict the MRR (Material removal rate). But the predictions have been quite off the mark. Even Linear Regression gave a much better result. I don't know where I'm going wrong. Below is my code in R:

data <-structure(list(A = c(50L, 50L, 50L, 50L, 50L, 60L, 60L, 60L, 
                            60L, 60L, 70L, 70L, 70L, 70L, 70L, 80L, 80L, 80L, 80L, 80L, 90L, 
                            90L, 90L, 90L, 90L), B = c(3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 
                                                       11L, 3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 
                                                       11L), C = c(100L, 200L, 300L, 400L, 500L, 200L, 300L, 400L, 500L, 
                                                                   100L, 300L, 400L, 500L, 100L, 200L, 400L, 500L, 100L, 200L, 300L, 
                                                                   500L, 100L, 200L, 300L, 400L), D = c(65L, 70L, 75L, 80L, 85L, 
                                                                                                        75L, 80L, 85L, 65L, 70L, 85L, 65L, 70L, 75L, 80L, 70L, 75L, 80L, 
                                                                                                        85L, 65L, 80L, 85L, 65L, 70L, 75L), E = c(0.2, 0.3, 0.4, 0.5, 
                                                                                                                                                  0.6, 0.5, 0.6, 0.2, 0.3, 0.4, 0.3, 0.4, 0.5, 0.6, 0.2, 0.6, 0.2, 
                                                                                                                                                  0.3, 0.4, 0.5, 0.4, 0.5, 0.6, 0.2, 0.3), MRR = c(8.926014, 14.10501, 
                                                                                                                                                                                                   38.40095, 48.49642, 88.21002, 4.892601, 15.179, 26.92124, 38.78282, 
                                                                                                                                                                                                   89.16468, 5.298329, 10.04773, 18.30549, 49.21241, 79.57041, 2.362768, 
                                                                                                                                                                                                   4.868735, 22.52983, 44.8926, 49.06921, 1.312649, 7.207637, 18.61575, 
                                                                                                                                                                                                   25.1074, 48.01909)), class = "data.frame", row.names = c(NA, 
                                                                                                                                                                                                                                                            -25L))

#Splitting the data

library(caTools)

set.seed(123)
split <- sample.split(data$MRR, SplitRatio = 0.7)
training_set <- subset(data, split == TRUE)
test_set <- subset(data, split == FALSE)

#Building the model and making predictions
library(randomForest)
set.seed(123)
rforest <- randomForest(x = training_set[-6],
                         y = training_set$MRR,
                         ntree = 500)

pred_rforest <- predict(rforest, test_set[,1:5])

#Also building a Decision tree model for the prediction
library(rpart)
dtree <- rpart(formula = MRR ~ .,
                  data = training_set,
                  control = rpart.control(minsplit = 1))

pred_dtree <- predict(dtree, test_set[,1:5])

#Checking the accuracy
library(MLmetrics)
MAPE(pred_dtree, test_set[,6])
MAPE(pred_rforest, test_set[,6])

I have even tried a larger dataset (with 100 training and 25 testing samples) but the accuracy where quite off the mark even there too. Both the decision tree and random forest results have been very bad.

Where I am going wrong? How to improve the results?

Any help would mean a lot.

$\endgroup$
3
  • 2
    $\begingroup$ Your data set is truncated so that we cannot see what it looks like. However, it seems like you have 5 independent variables. Random Forest works by taking random subsets of the variables.. Typically, this only makes sense when there are more variables than this. $\endgroup$ – G5W Mar 27 at 16:25
  • $\begingroup$ Can't it be viewed if you scroll horizontally? And come to think of it, that might be a problem in this RF model. The number of variables. But I have seen RF performing well even with 3 input parameters. $\endgroup$ – Shibaprasad Bhattacharya Mar 27 at 17:06
  • $\begingroup$ Your model is probably overfit: if you have less than 100 training instances, the model must be be very simple otherwise it's going to learn all the random details about the training data instead of generalizing. Did you try with one simple decision tree? Make sure that the hyper-parameters don't make the tree too complex (for instance the max depth should be low). $\endgroup$ – Erwan Mar 27 at 22:37
2
$\begingroup$

Doing simple bagging (mtry = 5) will give you RF predictive performance (RMSE = 14.3) somewhat close to the linear model (RMSE = 11.4). Modern machine learning methods are data hungry. You probably need a bigger dataset to see the RF outperform good ol' linear regression.

df <- structure(
    list(
        A = c(50L, 50L, 50L, 50L, 50L, 60L, 60L, 60L, 60L, 60L,
              70L, 70L, 70L, 70L, 70L, 80L, 80L, 80L, 80L, 80L,
              90L, 90L, 90L, 90L, 90L),
        B = c(3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 11L, 3L, 5L,
              7L, 9L, 11L, 3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 11L),
        C = c(100L, 200L, 300L, 400L, 500L, 200L, 300L, 400L, 500L,
              100L, 300L, 400L, 500L, 100L, 200L, 400L, 500L, 100L, 
              200L, 300L, 500L, 100L, 200L, 300L, 400L),
        D = c(65L, 70L, 75L, 80L, 85L, 75L, 80L, 85L, 65L, 70L, 85L,
              65L, 70L, 75L, 80L, 70L, 75L, 80L, 85L, 65L, 80L, 85L,
              65L, 70L, 75L),
        E = c(0.2, 0.3, 0.4, 0.5, 0.6, 0.5, 0.6, 0.2, 0.3, 0.4, 0.3, 
              0.4, 0.5, 0.6, 0.2, 0.6, 0.2, 0.3, 0.4, 0.5, 0.4, 0.5, 
              0.6, 0.2, 0.3),
        MRR = c(8.926014, 14.10501, 38.40095, 48.49642, 88.21002, 
                4.892601, 15.179, 26.92124, 38.78282, 89.16468, 
                5.298329, 10.04773, 18.30549, 49.21241, 79.57041,
                2.362768, 4.868735, 22.52983, 44.8926, 49.06921, 
                1.312649, 7.207637, 18.61575, 25.1074, 48.01909)
    ), 
    class = "data.frame",
    row.names = c(NA, -25L)
)

head(df)

y_hat <- numeric(nrow(df))
for (i in 1:nrow(df)) {
    lreg <- lm(MRR ~ ., data = df[-i, ]) # LOOCV
    y_hat[i] <- predict(lreg, newdata = df[i, ])
}
sqrt(mean((y_hat - df$MRR)^2))

set.seed(1234)

library(randomForest)

rf <- randomForest(MRR ~ ., data = df, mtry = 5)
sqrt(mean((rf$predicted - df$MRR)^2)) # out-of-bag predictions
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.