2
$\begingroup$

I should use the bagging (bootstrap aggregating) technique in order to train a random forest classifier. I read here the description of this learning technique, but I have not figured out how I initially organize the dataset.

Currently I first load all the positive examples and immediately after the negative ones. Moreover, positive examples are less than half of the negative ones, so by sampling from the dataset uniformly, the probability of obtaining a negative example is greater than that of obtaining a positive example.

How should I build the initial dataset? Should I shuffle the initial dataset containing positive and negative examples?

$\endgroup$
2
$\begingroup$

Bagging, as you said, means boostrap aggregating. Each tree has to be built on a different datasets obtained by boostrap.

If your former dataset has $N$ elements, you will have to sample with replacement $N$ elements from that dataset to obtain a bootstrap sample.

This is standard bagging. If the classes are unbalanced you might oversample the minority class or undersample the majority one. This is just one method, for example there is a paper by Chen and Breiman (RF creator).

About standard bootstrapping, you might just generate a random number between $1$ and $N$ to obtain one record of the bootstrap sample.

In the unbalanced case we might go for oversampling for example. In order to produce one record in the bootstrap sample you might flip a coin (random number in $\{0,1\}$): if head you choose a random record from the positive ones; if tail you choose a random one from negatives. The classes will be balanced in the bootstrap sample.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.