1
$\begingroup$

Can I somehow implement AdaBoost procedure over a weak classifier from another library? For example over SVM from libsvm, or over some neural network.

The idea of AdaBoost is that current weights of each sample influence learning step of gradient process. I have implemented AdaBoost over cascade correlation network, and liked the effect. Now I want to try other methods, but cant modify their code, and in some methods there is just no gradient process.

So my question is: how can technically weights of training samples can influence learning process?

The only idea I have is to form a new train set, in which samples with greater weights occur several times proportionally to current weights, but is looks weird.

Are there other ideas? Thanks!

UPD: some svm implementations like in python's scikit-learn support weighted train set

Improve this question
$\endgroup$
6
  • $\begingroup$ Have you taken a look at the actual statement of the AdaBoost algorithm? It includes an explicit form for the updated weights as a function of the previous step's weights and predictions. It's not written explicitly in terms of gradients and it's almost black-box-ready as it is, although I don't know anything about how it's implemented. $\endgroup$
    – shadowtalker
    Jan 7, 2015 at 20:25
  • $\begingroup$ Yes, sure, the question is how to apply the calculated weights to the training algorithm $\endgroup$
    – allchemist
    Jan 7, 2015 at 20:30
  • $\begingroup$ Sorry. The question was unclear and I didn't realize what you were asking until I saw Andy Jones' answer $\endgroup$
    – shadowtalker
    Jan 7, 2015 at 21:49
  • $\begingroup$ English is not my native, so i can be difficult to be understood $\endgroup$
    – allchemist
    Jan 7, 2015 at 21:54
  • $\begingroup$ Out of curiosity what are you using this for? $\endgroup$
    – shadowtalker
    Jan 7, 2015 at 21:59

1 Answer 1

Reset to default
0
$\begingroup$

If the library you're interested in supports weighted samples, use those. Here's how to weight samples in LibSVM. Most libraries I've encountered offer something similar.

If the library doesn't support weighted samples, then as you say you can either replicate datapoints (ugh) or you can dig into the code and add that functionality.

Improve this answer
$\endgroup$
0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.