1
$\begingroup$

I am using Random Forest to predict the MRR (Material removal rate). But the predictions have been quite off the mark. Even Linear Regression gave a much better result. I don't know where I'm going wrong. Below is my code in R:

data <-structure(list(A = c(50L, 50L, 50L, 50L, 50L, 60L, 60L, 60L, 
                            60L, 60L, 70L, 70L, 70L, 70L, 70L, 80L, 80L, 80L, 80L, 80L, 90L, 
                            90L, 90L, 90L, 90L), B = c(3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 
                                                       11L, 3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 
                                                       11L), C = c(100L, 200L, 300L, 400L, 500L, 200L, 300L, 400L, 500L, 
                                                                   100L, 300L, 400L, 500L, 100L, 200L, 400L, 500L, 100L, 200L, 300L, 
                                                                   500L, 100L, 200L, 300L, 400L), D = c(65L, 70L, 75L, 80L, 85L, 
                                                                                                        75L, 80L, 85L, 65L, 70L, 85L, 65L, 70L, 75L, 80L, 70L, 75L, 80L, 
                                                                                                        85L, 65L, 80L, 85L, 65L, 70L, 75L), E = c(0.2, 0.3, 0.4, 0.5, 
                                                                                                                                                  0.6, 0.5, 0.6, 0.2, 0.3, 0.4, 0.3, 0.4, 0.5, 0.6, 0.2, 0.6, 0.2, 
                                                                                                                                                  0.3, 0.4, 0.5, 0.4, 0.5, 0.6, 0.2, 0.3), MRR = c(8.926014, 14.10501, 
                                                                                                                                                                                                   38.40095, 48.49642, 88.21002, 4.892601, 15.179, 26.92124, 38.78282, 
                                                                                                                                                                                                   89.16468, 5.298329, 10.04773, 18.30549, 49.21241, 79.57041, 2.362768, 
                                                                                                                                                                                                   4.868735, 22.52983, 44.8926, 49.06921, 1.312649, 7.207637, 18.61575, 
                                                                                                                                                                                                   25.1074, 48.01909)), class = "data.frame", row.names = c(NA, 
                                                                                                                                                                                                                                                            -25L))

#Splitting the data

library(caTools)

set.seed(123)
split <- sample.split(data$MRR, SplitRatio = 0.7)
training_set <- subset(data, split == TRUE)
test_set <- subset(data, split == FALSE)

#Building the model and making predictions
library(randomForest)
set.seed(123)
rforest <- randomForest(x = training_set[-6],
                         y = training_set$MRR,
                         ntree = 500)

pred_rforest <- predict(rforest, test_set[,1:5])

#Also building a Decision tree model for the prediction
library(rpart)
dtree <- rpart(formula = MRR ~ .,
                  data = training_set,
                  control = rpart.control(minsplit = 1))

pred_dtree <- predict(dtree, test_set[,1:5])

#Checking the accuracy
library(MLmetrics)
MAPE(pred_dtree, test_set[,6])
MAPE(pred_rforest, test_set[,6])

I have even tried a larger dataset (with 100 training and 25 testing samples) but the accuracy where quite off the mark even there too. Both the decision tree and random forest results have been very bad.

Where I am going wrong? How to improve the results?

Any help would mean a lot.

Improve this question
$\endgroup$
3
  • 2
    $\begingroup$ Your data set is truncated so that we cannot see what it looks like. However, it seems like you have 5 independent variables. Random Forest works by taking random subsets of the variables.. Typically, this only makes sense when there are more variables than this. $\endgroup$
    – G5W
    Mar 27, 2021 at 16:25
  • $\begingroup$ Can't it be viewed if you scroll horizontally? And come to think of it, that might be a problem in this RF model. The number of variables. But I have seen RF performing well even with 3 input parameters. $\endgroup$
    – Single Handed Sailor
    Mar 27, 2021 at 17:06
  • $\begingroup$ Your model is probably overfit: if you have less than 100 training instances, the model must be be very simple otherwise it's going to learn all the random details about the training data instead of generalizing. Did you try with one simple decision tree? Make sure that the hyper-parameters don't make the tree too complex (for instance the max depth should be low). $\endgroup$
    – Erwan
    Mar 27, 2021 at 22:37

1 Answer 1

Reset to default
2
$\begingroup$

Doing simple bagging (mtry = 5) will give you RF predictive performance (RMSE = 14.3) somewhat close to the linear model (RMSE = 11.4). Modern machine learning methods are data hungry. You probably need a bigger dataset to see the RF outperform good ol' linear regression.

df <- structure(
    list(
        A = c(50L, 50L, 50L, 50L, 50L, 60L, 60L, 60L, 60L, 60L,
              70L, 70L, 70L, 70L, 70L, 80L, 80L, 80L, 80L, 80L,
              90L, 90L, 90L, 90L, 90L),
        B = c(3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 11L, 3L, 5L,
              7L, 9L, 11L, 3L, 5L, 7L, 9L, 11L, 3L, 5L, 7L, 9L, 11L),
        C = c(100L, 200L, 300L, 400L, 500L, 200L, 300L, 400L, 500L,
              100L, 300L, 400L, 500L, 100L, 200L, 400L, 500L, 100L, 
              200L, 300L, 500L, 100L, 200L, 300L, 400L),
        D = c(65L, 70L, 75L, 80L, 85L, 75L, 80L, 85L, 65L, 70L, 85L,
              65L, 70L, 75L, 80L, 70L, 75L, 80L, 85L, 65L, 80L, 85L,
              65L, 70L, 75L),
        E = c(0.2, 0.3, 0.4, 0.5, 0.6, 0.5, 0.6, 0.2, 0.3, 0.4, 0.3, 
              0.4, 0.5, 0.6, 0.2, 0.6, 0.2, 0.3, 0.4, 0.5, 0.4, 0.5, 
              0.6, 0.2, 0.3),
        MRR = c(8.926014, 14.10501, 38.40095, 48.49642, 88.21002, 
                4.892601, 15.179, 26.92124, 38.78282, 89.16468, 
                5.298329, 10.04773, 18.30549, 49.21241, 79.57041,
                2.362768, 4.868735, 22.52983, 44.8926, 49.06921, 
                1.312649, 7.207637, 18.61575, 25.1074, 48.01909)
    ), 
    class = "data.frame",
    row.names = c(NA, -25L)
)

head(df)

y_hat <- numeric(nrow(df))
for (i in 1:nrow(df)) {
    lreg <- lm(MRR ~ ., data = df[-i, ]) # LOOCV
    y_hat[i] <- predict(lreg, newdata = df[i, ])
}
sqrt(mean((y_hat - df$MRR)^2))

set.seed(1234)

library(randomForest)

rf <- randomForest(MRR ~ ., data = df, mtry = 5)
sqrt(mean((rf$predicted - df$MRR)^2)) # out-of-bag predictions
Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.