0
$\begingroup$

I have a binary classification dataset with all features in numeric form. Most of them are continuous variable (within ranges, for example - from 2.50001 to 2.9999).

I want to predict the output which is a binary classification (0 or 1).

I used many classification algorithm but decision tree works the best. I would like to know why this is good and why others are bad? There is a little difference between the training and testing accuracies for all the algorithms, so there wont be any overfitting.

enter image description here

I would like to hear some experts advices here. Thanks in advance.

Improve this question
$\endgroup$
2

1 Answer 1

Reset to default
1
$\begingroup$

Decision tree are extremely efficient, very easy to tune and can handle non-linearity extremely well.

Adaboost and RandomForest should be better than Decision Tree but they might be overfitting a bit.

If the exemple of feature is correct and you have that many precision digits, then all the more for Trees because they can split on specific values that would be characteristic of your binary outcome. For instance they could split 2.50001 from 2.50000.

Now Neural Network can be as good as trees but they are a lot harder to fine tune and that could explain your result. Usually on small to medium tabular data, Decision Tree (Xgboost for intance) will outperform Neural Net. Or put it differently it would take a very large amount of time to fine tune NN to match Gradient Boosted trees.

Finally, SVM is very inefficient and linear by default. So except if you set the kernel to a non-linear one like RBF, this result is completely expected.

Improve this answer
$\endgroup$
5
  • $\begingroup$ Wow. Thanks a lot. $\endgroup$
    – JSVJ
    Commented Mar 30, 2021 at 10:10
  • $\begingroup$ Happy it answers your question. Could you note my post as the correct answer ? thanks you :) $\endgroup$
    – Romain
    Commented Mar 30, 2021 at 10:12
  • $\begingroup$ Could you advice me one more thing. Why would the decision tree give different accuracies everytime we run it using the same data. $\endgroup$
    – JSVJ
    Commented Mar 30, 2021 at 10:22
  • 2
    $\begingroup$ Hard to answer without your code, but there are some implementation that are non deterministic (include randomness in the selection of column, splits etc...) so each run could be slightly different $\endgroup$
    – Romain
    Commented Mar 30, 2021 at 10:24
  • $\begingroup$ Thanks. I understood now $\endgroup$
    – JSVJ
    Commented Mar 30, 2021 at 10:31

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.