1
$\begingroup$

First of all, I am very new to this subject. I have been using weka some days to familiarize myself and reading about data mining to understand it.

I have a dataset (from UCI, german credit data) and I need to classify it. I have been trying different tree algorithms using different attributes instead of all and I wish to find which one gives me the best tree. (splitting my data 66/34 for training and testing)

Is there a way to automate this, that is make weka try all combinations of attributes to find which ones give the best tree? I would prefer to use weka since is the tool I have been using, but this is not compulsory.

$\endgroup$

1 Answer 1

1
$\begingroup$

My interest in machine learning started with weka, but very soon after I switched to R, because it's more hands-on than using the explorer in weka.

In regard to classification trees, this is a huge subject, and as such there is no "best" algorithm that works straight out of the box. You will have to play with different models, understand theoretically what are the pros and cons, and how they overcome deficiencies in simpler models. Having said that, if you can get a simple model to work, it would be better to have than a more complex model.

One of the basic algorithms for classification trees is CART and one of the more advanced models is Random Forest. This latter model has a feature to select a subset of your attributes.

A good place to start would be link. It's is written for R user, but the same models exist in weka if u prefer.

$\endgroup$
3
  • $\begingroup$ randomforest looks interesting. I tried it in weka and R but I could not find how to get the best found tree (like you do when using J48). could you provide a bit more information? $\endgroup$ Nov 18, 2011 at 13:11
  • $\begingroup$ Hi Santiagozky: there is no single "best" tree that the random forest algorithm produces. In fact this algorithm uses multiple trees, and hence the name "forest". For details explanation of the algorithm, i suggest the following link $\endgroup$
    – Lalas
    Nov 19, 2011 at 3:42
  • $\begingroup$ thank you very much. I thought that random forest allows me to get a single set of rules to classify. since that is what I need I will go for CART. $\endgroup$ Nov 21, 2011 at 21:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.